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PREFACE 

Every  administrator  bears  a  primary  responsibility  for  planning  the 
accomplishment  of  those  objectives  and  activities  that  fall  in  his  area 
of  responsibility.  When  responsibility  is  relatively  restricted  in 
extent,  an  administrator  can  well  use  the  informal  and  intuitive  methods 
that  constitute  "sound  judgement;"  however,  as  his  area  of  responsibility 
widens  to  include  tasks  of  increasing  complexity,  he  may  feel  that  it 
would  be  useful  to  implement  more  formal  methods  of  planning  and  control. 
When  the  administrator  is  responsible  for  planning  or  controlling  whole 
systems  of  activities  (such  as  a  school  district)  or  when  the  activities 
are  themselves  new  or  experimental  or  when  large  sums  of  money  are  to 
be  expended,  then  the  use  of  one  or  another  formal  planning  process  may 
become  necessary  in  order  to  maintain  the  desired  degree  of  control. 

A  vast  literature  exists  on  methods  of  planning  for  systems,  for 
"comprehensive  planning."   It  is  hoped  that  those  who  are  involved  in 
system-wide  planning  will  find  something  of  use  to  them  in  this  publi- 
cation. 

The  first  section  discusses  and  evaluates  many  of  the  ideas  and 
methods  that  are  to  be  found  in  the  contemporary  literature  on  compre- 
hensive planning.   Since,  in  this  young  field  of  study,  accepted 
standards  for  evaluation  of  various  ideas  and  methods  do  not  exist,  it 
must  be  confessed  that  the  author's  subjective  judgements  have  of  neces- 
sity influenced  the  presentation. 
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An  annotated  bibliography  constitutes  the  second  section.  The 
items  there  have  been  selected  by  the  author  for  their  potential  use- 
fulness to  those  responsible  for  comprehensive  planning.  Often  an 
abstract,  prepared  by  the  author  of  an  item,  was  available.   In  those 
cases  the  author's  abstract  was  accepted  as  an  annotation.  Credit  for 
the  remaining  annotations  belongs  to  Lewis  Polin.   Joan  Rosenstein  and 
Elenore  Pritchard  provided  editorial  assistance.  Loretta  Margulies  and 
Shakuntala  Asnani  handled  typing  and  logistics  for  the  project  in  their 
usual  efficient  manner. 

Sanford  Temkin 
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ORGANIZATION  AND  USE  OF  THIS  BIBLIOGRAPHY 

The  arrangement  of  this  bibliography  is  classified  into  five  general 
categories.  The  categories  are  1.0,  PLANNING  PROCESS  INFORMATION;  2.0, 
SYSTEM  WIDE  PLANNING  METHODS;  3.0,  SYSTEM  CHARACTERISTICS  AND  QUALITIES; 
4.0,  POLITICS,  COMMUNITY,  IMPLEMENTATION  AND  COMMUNICATION;  5.0,  BIBLI- 
OGRAPHY.  Each  category  is  subdivided,  e.g.,  4.1  is  Politics  and  Community 
and  4.2  is  Politics  -  Implementation.  Within  each  subdivision  the  ref- 
erences are  listed  alphabetically  by  author.  Author  and  subject  indexes 
follow  the  annotated  bibliography  section.  The  author  index  refers  the 
reader  to  the  appropriate  page  number  in  the  annotated  bibliography 
section.  The  subject  index  indicates  the  item  number  where  the  subject 
can  be  found,  e.g.,  4.1.2  indexes  the  second  article  of  subdivision  4.1. 

Fleur  Weinberg,  Librarian 


-5-     CPL  Exchange  Bibliography  #219 

PAGE 

PREFACE  2 

ORGANIZATION  AND  USE  OF  THIS  BIBLIOGRAPHY 4 

AN  EVALUATION  OF  THE  LITERATURE  OF  COMPREHENSIVE  PLANNING   .  7 

1.0  PLANNING  PROCESS  INFORMATION   .  19 

Pupil  Needs 19 

Overall  Objectives.   .     21 

Assignment  of  Weights  to  Objectives  and  Outcomes.   .   .  22 

Performance  Criteria  29 

Cost  and  Accounting  Problems 33 

2.0  SYSTEM  WIDE  PLANNING  METHODS   3G 

Comprehensive  Planning 35 

PPBS •:• 

General  Systems  and/or  Economic  Analysis      40 

Cost-Effectiveness  Analysis        45 

Input-Output  Analysis 48 

Cost/Benefit  analysis        50 

3.0      SYSTEM  CHARACTERISTICS  AND  QUALITIES        52 

Feasibility 52 

Management  Information  Systems        53 

Feedback 56 

Organizational  Structure   56 

4.0  POLITICS,  COMMUNITY,  IMPLEMENTATION  AND  COMMUNICATION  .  60 

Politics  and  Community   60 

Implementation  ' 

Communication  69 


-6-     CPL  Exchange  Bibliography  #219 

PAGE 

5.0  BIBLIOGRAPHY 71 

AUTHOR  INDEX 75 

SUBJECT  INDEX   31 


CPL  Exchange  Bibliography  #219 


AN  EVALUATION  OF  THE  LITERATURE 
OF  COMPREHENSIVE  PLANNING 


The  term  "comprehensive"  has  meanings  which  are  easily  confused. 
On  the  one  hand,  it  is  always  possible  to  include  more  information  in 
the  planning  process  and  thus  label  the  process  as  more  comprehensive. 
On  the  other  hand,  the  word,  may  be  employed  in  a  completely  different 
sense.   But  before  considering  this  other  meaning  it  is  necessary  to 
define  the  term  "system."  A  system  may  be  defined  as  a  set  of  mutually 
dependent  activities  which  is  directed  towards  a  set  of  objectives 
common  to  all  those  activities.  The  term  comprehensive  in  its  second 
sense  refers  to  a  planning  process  which  determines  and  evaluates  alter- 
native decisions  within  the  total  context  of  the  set  of  shared  objectives. 
As  used  in  this  monograph  the  term  comprehensive,  then,  does  not  denote 
a  concern  for  more  information  about  a  school  district  per  se,  but  a 
concern  for  that  information  which  allows  one  to  view  a  district's 
operations  as  being  directed  toward  a  set  of  objectives  common  to  all 
the  activities  of  the  district. 

The  indispensible  elements  of  a  process  are  called  "necessary  and 
sufficient  conditions."  Planning  processes  have  indispensible  types  of 
information.  The  nature  and  specific  interrelationships  among  these 
types  of  information  depend  upon  what  the  planning  process  is  designed 
to  achieve.  There  are  several  ways  to  attempt  to  identify  the  types  of 
data  that  are  indispensible  for  a  comprehensive  planning  process.   The 
first  approach  selected  by  this  writer  was  to  use  mathematics  to  select 
the  types  of  data  necessary  to  achieve  the  best  possible  fulfillment  of 
all  the  objectives  of  the  system,  given  the  restraints  imposed  by  the 
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resources  available.  Another  way  to  identify  indispensible  types  of 
information  would  be  to  read  comprehensive  planning  literature  in  the 
hope  that  they  could  be  recognized  empirically  and  intuitively.  Yet 
another  way  would  be  to  consult  with  experienced  planners  and  planning 
theoreticians.   In  actuality,  all  three  approaches  influenced  this 
paper.  The  types  of  data  that  are  needed  for  a  comprehensive  planning 
process  are: 

1.  A  set  of  planning  objectives  (or  overall  objectives)  of  the 
whole  system. 

2.  Information  about  the  relative  importance  ("weight")  that 
various  people  involved  in  the  system  believe  is  appropriate 
for  the  different  planning  objectives. 

3.  At  least  one  indicator  of  how  the  degree  of  attainment  of  each 
objective  is  to  be  judged  (a  "performance  indicator"). 

4.  Information  about  the  potential  importance  of  various  programs 
and  activities  to  each  of  the  planning  objectives  (an  approxi- 
mation to  the  concept  of  the  "production  function"). 

5.  Performance  outcomes  for  each  performance  indicator. 

6.  Costs. 

There  are  two  considerations  that  arise  when  one  discusses  these  types 
of  data.  First  there  are  questions  about  measurement  and  data  collection 
techniques.  Second  there  are  questions  about  relationships  among  types 
of  data,  i.e.,  the  synthesis,  or  how  to  "put  it  all  together."  Infor- 
mation types  are  discussed  in  these  two  regards.  References  are  made 
to  particular  bibliographical  items  that  are  annotated  in  the  main  body 
of  this  monograph. 
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Page  numbers  refer  to  the  reference's  page  number  in  the  annotated 

bibliography. 

Planning  objectives  or  overall  objectives 

If  one  visualizes  a  line  that  has  vague  goals  at  the  extreme  left 
and  behavioral  objectives  at  the  extreme  right,  planning  objectives  are 
found  "about  2/10"  of  the  way  from  the  left  extreme  (see  sketch). 


Planning 
Objectives 


Vague  Behavioral 

Goals  Objectives 


Planning  objectives  are  a  higher  order  abstraction  in  relation  to  very 
specific  instructional  objectives  and  as  such  play  a  different  role  in 
planning.  Planning  objectives  have  a  single  function  -  to  allow  the 
operations  of  the  school  district  to  be  placed  in  a  network  of  relation- 
ships for  subsequent  planning  and  discussion.  This  approach  contrasts 
with  one  that  is  somewhat  widely  held  -  namely  that  objectives  to  be 
useful  must  be  measurable.   It  is  true  that  measurement  is  important; 
but,  measurement  should  be  reserved  for  the  criteria  of  performance, 
which  have  the  function  of  determining  the  extent  to  which  a  given 
activity  has  done  what  it  was  intended  to  do  for  the  system  as  a  whole. 

The  Downey  study  (p.  21)  has  the  most  to  say  about  goals  and  objec- 
tives; the  Educational  Testing  Service  study  (p.  21),  prepared  for  the 
Pennsylvania  Department  of  Education,  has  generated  a  great  deal  of  per- 
formance data  for  "ten  goals  of  education"  since  1965;  and  the  Radnor 
paper  (p.  22)  represents  a  recent  statement  by  a  small  suburban  distric. 
that  is  interested  in  objectives,  performances,  and  planning  for  change. 
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Assignment  of  weights  to  planning  objectives 

As  one  reads  about  decision-making  and  listens  to  decision-makers, 
a  clear  message  emerges  -  priorities  are  all  important  in  planning.  This 
writer  must  disagree  with  certain  implications  of  this  conclusion.   De- 
ciding on  priorities  may  be  important  to  "get  a  feel  for  things"  but  they 
are  not  all  important  in  the  planning  process.   For  instance,  picture  a 
decision-maker  with  three  objectives  which  have  been  ranked  in  terms  of 
their  priorities.   Does  he  allocate  resources  exclusively  to  the  highest 
priority  objective':   If  not.  how  much  does  he  allocate  to  the  highest 
priority  objective  before  he  allocates  resources  to  the  middle  priority 
objective?  Or  does  he  allocate  some  resources  to  each  objective?   If 
so,  how? 

These  questions  can  be  answered  partially  by  an  assignment  of  rela- 
tive weights  to  the  planning  objectives.  Once  this  notion  is  accepted 
then  a  host  of  other  questions  emerge.   Seme  of  these  are: 

1.  How  can  weights  be  obtained  from  people? 

2.  Can  weights  be  obtained  from  both  individuals  and  groups? 

3.  Are  these  weights  really  measurements? 

4.  How  do  the  weights  obtained  from  one  person  compare  to  those 
obtained  from  another? 

There  are  three  groups  of  methods  for  obtaining  assignment  of 
relative  importance  or  preference  from  people.  The  Churchman- Ackoff 
Method  may  be  applied  to  individuals  or  groups.   The  method  forces  the 
subject  to  consider  and  reconsider  the  relative  importance  of  objectives 
singularly  and  in  combination.  Hopefully,  the  subject  is  able  to 
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generate  a  set  of  internally  consistent  assignments  of  importance  to  each 
objective.  This  method  and  variations  of  it  are  common  in  the  literati.  :  - 
see,  for  instance,  Churchman,  Ackoff  and  Arnoff  (p.  24);  and  Stimson  (p.  2  '<), 

The  Neumann-Morgenstern  Cardinal  Utility  approach  has  been  used 
primarily  by  economists  and  statisticians.   It  centers  on  an  individual's 
responses  to  selected  proposals.  These  proposals  are  usually  stated  in 
terms  of  dollar  return  and  probability  of  accomplishment.  A  good  descrip- 
tion of  the  method  can  be  found  in  Baumol  (p.  23);  while  a  worthwhile 
empirical  study  of  individual  utility  functions  can  be  found  in  Grayson's 
classic  study  (p.  25)  of  decision-making  practices  in  the  oil-well  drilling 
industry. 

The  Delphi  Method  first  appeared  in  the  literature  around  1963. 
Essentially,  it  is  designed  for  group  judgements  and  arrives  at  consensus 
by  using  a  series  of  steps.   In  each  step  subjects  are  told  the  results 
of  judgements  made  at  the  end  of  the  previous  step.  Those  people  who 
maintain  extreme  judgements  justify  their  positions  for  others  to  evaluate. 
By  this  procedure  consensus  is  ultimately  achieved.   Although  this  im 
is  currently  extremely  popular  very  little  had  been  done  to  explore  its 
assumptions  until  recently.   Norman  Dalkfiy,  one  of  the  Delphi  innovators, 
recently  has  discussed  some  of  these  assumptions  (p.  24),  while  Rescher 
(p.  27)  provides  additional  insight  into  the  subtleties  of  the  method. 

Since  judgements  of  relative  importance  (weights)  obtained  from  on.2 
person  cannot  be  added  or  subtracted  from  those  obtained  from  another, 
inter-personnel  utility  comparisons  cannot  be  handled  directly.  The 
best  solution  would  seem  to  be  that  of  avoiding  summing  individual 
utility  functions.  Accordingly,  any  evaluation  of  alternatives  for 
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purposes  of  allocating  resources  would  be  done  for  a  specific  set  of 
weighted  planning  objectives.  This  would  mean  that  each  time  a  set  of 
preferences  is  introduced  a  new  allocation  of  resources  would  result. 
When  the  preferences  of  many  individuals  or  groups  are  to  be  explicitly 
considered,  computer  processing  is  undoubtedly  required. 

The  Churchman-Ackoff  Method  claims  to  meet  the  basic  additivity  and 
transitivity  assumptions  for  measurement.  Baumol  indicates  Neumann- 
Morgenstern  indexes  also  meet  these  requirements.  As  was  indicated 
earlier  there  is  still  much  to  be  learned  about  the  Delphi  Method. 

This  author  feels  that  measurement  theory  and  methodology  lag  far 
behind  the  need  for  practical  real-world  methods. 

Another  domain  which  may  be  addressed  with  regard  to  preference 
assignments  is  that  of  pupil  needs,  an  area  in  which  little  has  been  done. 
Standard  economic  theory  indicates  that  the  best  distribution  of  budgetary 
funds  among  all  activities  occurs  when  the  additional  usefulness  of  each 
dollar  for  an  individual  are  proportional  to  the  prices  he  pays.  These, 
so  called,  "marginal  utilities"  are  assumed  to  represent  a  situation  in 
which  the  individuals  needs  are  satisfied.  School  districts  also  have 
a  budget  to  allocate  and  it  would  seem  reasonable  to  take  into  considera- 
tion pupil  needs.  As  one  examines  the  literature  there  is  frequent  mention 
of  "meeting  the  needs  of  kids"  but  there  is  very  little  to  guide  someone 
on  how  to  assess  these  needs.   In  this  author's  opinion  the  most  sensible 
approach,  is  the  study  by  Campbell  and  Markle  at  the  Far  West  Laboratory 
for  Educational  Research  and  Development  (p.  19)  and  Mirsky's  work  on 
"pupil-event  analysis"  (p.  20). 
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Hopefully,  data  on  pupil  needs  would  serve  as  input  into  the  process 
by  which  assignments  of  relative  importance  to  overall  objectives  would 
be  determined  and  represented.   It  is  argued  by  this  author,  that  perhaps 
a  meaningful  arrangement  can  emerge  as  various  community  groups  discuss 
and  debate  the  relative  importance  of  different  pupil  needs  and  their 
implications  for  the  school  district's  planning  objectives. 

Performance  criteria  or  indicators 

Just  as  there  are  many  ways  to  think  about  objectives,  one  encounters 
many  ways  to  assess  performance.  A  generalized  understanding  of  perform- 
ance criteria,  in  this  author's  experience,  results  from  the  notion  of 
"operational  criteria."  Although  operational  criteria  are  arbitrary  and 
subjective,  they  explicitly  state  what  will  be  observed,  by  whom,  when, 
and  under  what  conditions.  Behavioral  criteria,  a  sub-class  of  opera- 
tional criteria,  are  designed  for  observing  behavior  and  are  most  useful 
for  assessing  behavioral  objectives.  Assessment  of  performance  for 
planning  objectives  may  or  may  not  necessitate  direct  behavioral  obser- 
vation. 

This  author  partitions  performance  assessment  into  four  steps: 

1.  The  decision-maker  must  specify  a  performance  criterion.  A 
performance  criterion  specifies  what  will  be  observed  to 
determine  the  extent  to  which  a  given  objective  has  been  met. 

2.  It  is  determined  who  will  do  the  observing,  when  and  under  what 
conditions. 

3.  Possible  performance  outcomes  are  anticipated  and  their  merit 
interpreted.  Mathematically,  we  can  say,  an  X-axis  would  display 
various  outcomes  and  a  Y-axis  would  show  the  respective  values 

as  a  percentage  (0  -  100). 
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4.  The  final  step  involves  making  the  observations  and  using  the 
index  to  interpret  the  merit  (or  value)  of  the  outcome  that  did 
result. 

The  basic  idea  underlying  a  performance  indicator  is  that  a  knowl- 
edgeable decision-maker  is  able  to  specify,  given  some  assistance  and 
support,  what  information  he  would  like  to  evaluate  in  order  to  deter- 
mine the  extent  to  which  a  given  overall  objective  has  been  served.  An 
interesting  characteristic  of  a  performance  indicator,  defined  in  the 
manner  suggested,  is  that  two  children  who  receive  the  same  item  score 
(each  get  four  correct  out  of  a  total  of  eight  responses)  may  be  credited 
with  different  performance  index  outcomes,  depending  on  the  assigned 
importance  of  that  score  to  each  individual  child's  progress. 

Effectiveness 

Concepts  of  effectiveness  have  been  applied  in  various  ways  in 
planning.  Usually  the  term  is  used  as  synonymous  with  good  performance. 
A  few  authors  indicate  that  effectiveness  may  be  viewed  as  a  function  of 
two  variables  —  performance  and  the  potential  of  the  respective  activities 
to  contribute  to  the  overall  organization  or  system.  An  assessment  of 
the  ongoing  system,  as  it  is  operating,  can  be  made  by  considering 
effectiveness  as  a  function  of  performance  and  worth.  The  requirements 
for  such  an  evaluation  are: 

1.  Each  overall  objective  is  assigned  a  weight  in  accord  with 
its  importance  relative  to  the  other  overall  objectives. 

2.  Each  activity  conducted  under  the  auspices  of  the  ongoing 
system  is  assigned  a  weight  in  accord  with  its  relative 
importance  or  potential  to  contribute  to  its  overall  objective(s). 
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This  allows  the  relative  worth  of  each  overall  objective  to  be 

partitioned  among  its  component  activities. 
3.  A  performance  index,  which  will  give  a  more  or  less  formalized 

indication  of  the  extent  to  which  the  activity  has  done  what  it 

could  do,  has  been  designed  for  each  activity. 
When  activity  performance  is  perfect  (1.0,  or  100%  in  other  words,  on 
the  performance  index)  the  full  worth  of  the  activity  is  realized  and  it 
is  said  to  be  totally  effective.  The  effectiveness  of  an  activity,  in 
this  context,  is  its  performance  index  outcome  weighted  by  its  potential 
for  contributing  to  the  system.  This  implies  that  if  two  activities  per- 
form equally,  the  one  with  the  higher  potential  value,  from  the  point  of 
view  of  a  system-wide  evaluation,  is  more  effective. 

Authors  who  give  considerable  attention  to  the  synthesis  of  concepts 
dealing  with  objectives,  values  and  performance  are  Fallon  (p.  25), 
Chuang  and  Peper  (p.  35),  and  Temkin  (p.  36). 

Most  authors  accept  the  notion  that  output  of  a  system  should  in 
some  sense  be  related  to  input  so  that  alternatives  can  be  evaluated. 
One  family  of  such  methods  comes  from  the  "input-output"  methods  of 
Wassily  Leontief.*  The  shortcomings  of  such  applications  are  that  they 
avoid  consideration  of  the  process  by  which  different  sets  of  inputs 
relate  to  different  outputs.   In  applications  to  education,  process 
variables  are  generally  mixed  with  input  variables  and  those  related 
to  characteristics  of  the  child  and  the  home.  But  were  the  variables 


*W.W.  Leontief,  The  Structure  of  American  Economy,  1919-1939, 
New  York,  1951,  2nd  edition. 
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unconfounded  there  would  still  remain  the  problem  of  relating  input- to- 
process-combinations  to  process-to-output-combinations.  One  of  the  most 
interesting  methods  belongs  to  Dyer  (p.  48),  who  suggests  a  scheme  that 
combines  input-output  search  for  hypotheses  with  experimental  design 
considerations.  Were  it  not  for  a  scarcity  of  funds  for  education,  Dyer's 
method  could  make  a  practical  contribution  to  educational  decision-making, 
since  it  would  have  the  capability  of  taking  educational  processes  into 
consideration. 

Another  way  to  bring  input,  process  and  output  information  together 
would  be  to  develop  a  series  of  curves,  one  for  each  activity,  that 
relate  effectiveness  to  cost.  This  information  would  allow  one  to  evalu- 
ate the  amount  of  change  in  effectiveness  for  each  change  in  cost  for  an 
ongoing  activity.  This  could  be  plotted  as  a  cost-effectiveness  curve. 
Proposed  alternatives  could  be  introduced  by  developing  a  new  curve  and 
recognizing  that  new  activities  often  have  associated  new  fixed  costs. 
Once  the  curves  are  drawn,  then  mathematical  methods  such  as  linear 
programming  or  a  non-linear  adaptation  (see  McNamara,  p.  44  or  Sisson's 
comments  in  Temkin's  Appendix,  p.  47)  could  be  used  to  search  for  pre- 
ferred alternatives. 

Synthesis  and  implementation 

PPBS-type  methods  were  originally  designed  for  research  and  develop- 
ment organizations.  This  planning  mechanism  was  used  to  help  select 
from  among  alternative  ways  to  produce  a  given  capability.  The  process 
generally  began  by  defining  a  set  of  capabilities  or  "missions."  This 
was  followed  by  the  development  of  alternative  programs  and  budgets  that 
could  support  the  respective  alternatives.  Generally  a  multi-year  plan 
resulted. 
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One  of  the  major  needs  in  managing  a  school  system  is  a  technique 
capable  of  relating  outputs  to  costs.  This  need  is  dramatized  by  the 
increasing  demands  on  a  small  supply  of  educational  dollars. 

A  PPBS  is  such  an  approach.  The  method,  however,  is  more  applicable 
to  the  planning  of  a  completely  new  school  system  than  to  planning  changes 
for  an  existing  school  system. 

A  fundamental  problem  in  applying  PPBS  to  schools  is  that  it  does 
not  lend  itself  to  orderly  transition  from  where  a  school  district  is  to 
where  management  wants  it  to  be.  Orderly  systemic  change  is  a  dynamic 
systems  engineering  problem  that  does  not  lend  itself  to  a  situation  in 
which  ends  are  merely  specified  and  necessary  changes  are  assumed  to  just 
happen  automatically  once  this  is  done.  People  are  responsible  for  con- 
ducting the  affairs  of  schools.  These  people  are  individuals  whose 
present  behaviors  are  based  on  attitudes,  values,  beliefs,  customs, 
training  and  experiences.  Croston  and  Gregory  (p.  66)  indicate  that  the 
usual  view  of  an  organization,  from  within,  supports  a  situation  in  which 
necessary  information  tends  to  flow  upwards  and  downwards,  but  if  plan- 
ning information  is  to  be  related  to  organizational  objectives  a  horizontal 
flow  of  information  is  needed.   Helping  people  within  the  organization  to 
understand  and  become  comfortable  with  organizational  objectives  and 
horizontal  communication  flows  is  a  necessary  condition  for  comprehensive 
planning.  Other  necessary  conditions  are  executive  commitment  (see  Benton 
and  Tenzer  p.  65);  widespread,  meaningful  involvement  in  planning  and 
decision-making  (see  Eberle  p.  66);  sometimes,  organizational  restructuring 
in  order  to  facilitate  planning  and  change  (see  Andes  and  Kimbrough  p.  56, 
and  Maguire  p.  59);  and  a  good  management  system  once  intended  changes 
are  specified  (see  Cook  p.  66). 
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The  essence  of  a  comprehensive  planning  process  for  schools,  in  the 
opinion  of  this  writer,  is  to  show  teachers  how  to  look  at  their  work 
in  the  classroom  so  that  they  can  evaluate  what  they  do.   In  addition, 
roles  are  to  be  defined  and  clarified  for  principals,  central  office 
administrators  and  community  groups.   Effective  and  efficient  change 
seldom  comes  about  by  means  of  changes  in  the  accounting  system  or  the 
budget  alone. 

In  an  ongoing,  dynamic,  and  changing  system  the  superintendent  can 
detect  changes  in  pupil  achievements  and  changes  in  program  budgets. 
But  the  simple  fact  remains  that  meaningful  change  can  be  best  accomplished 
by  developing  and  maintaining  an  educational  process  that  is  responsive 
to  changing  needs. 

The  above  mentioned  methods,  techniques,  and  ideas  can,  and  already 
have,  contributed,  in  various  measure  and  with  varying  results,  to  the 
problem  of  comprehensive  system-wide  planning  in  education.  The  following 
bibliography  represents,  hopefully,  the  vast  emerging  literature  on  the 
subject. 
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1.0  PLANNING  PROCESS  INFORMATION 

1.1  Pupil-Needs 

1)  Campbell,  Vincent  N. ,  and  David  G.  Markle.  "Identifying  and 
Formulating  Educational  Problems."  Berkeley,  Calif.: 
Far  West  Laboratory  for  Educational  Research  and  Develop- 
ment, April,  1968. 

"The  aim  of  this  project  was  to  develop  effective  techniques 
for  identifying  educational  needs  and  formulating  them  into 
well  defined  problems.  The  critical  incident  technique  was 
used  to  identify  need  symptoms  perceived  by  educators  for  a 
representative  sample  of  schools  throughout  the  region 
served  by  the  Far  West  Laboratory.  A  hierarchy  of  categories 
was  developed  to  describe  these  need  data,  which  were  then 
classified  and  tabulated  by  category.  The  problem  formu- 
lation techniques  explored  informally  included  interview, 
observation,  and  small  group  discussion.  We  decided  that 
knowledge  of  problem  defining  behavior  is  too  primitive  to 
justify  step-by-step  systemization  of  the  process  of  formu- 
lation. The  techniques  we  developed  rather  relied  heavily 
on  intuitive  processes,  and  systematic  control  was  aimed 
toward  insuring  that  the  product  of  the  formulation  contained 
the  following  elements:  outcomes  desired,  values  under- 
lying outcomes,  kinds  of  evidence  for  outcomes,  present 
conditions,  solution  possibilities  and  immediate  action 
alternatives.  We  experimentally  compared  three  group 
problem  defining  techniques  which  differed  on  two  dimen- 
sions: (a)  whether  or  not  the  formulation  was  structured 
to  obtain  the  elements  listed  above,  and  (b)  whether  or 
not  the  inquiry  was  directed  by  an  experienced  outside  formu- 
lator.   Eighteen  three-person  groups  of  educators  were 
used  in  all.  The  problem  definition  produced  by  each  group 
was  evaluated  independently  by  two  other  educators  from 
the  same  district.  Results  favored  the  directed  groups; 
among  the  undirected  groups  completely  unstructured  groups 
rated  as  high  or  higher  than  groups  producing  a  structured 
definition.   Implications  for  training  and  promising 
directions  for  further  work  on  problem  formulation  systems 
are  discussed."  (Author) 

2)  McElhinney,  James  H.,  Richard  C.  Kunkel,  and  Lawrence  A. 
Lucas.   "Evidences  of  School  Related  Alienation  in 
Elementary  School  Pupils."  Paper  presented  at  the  Ameri- 
can Educational  Research  Association,  Minneapolis,  March, 
1970. 

The  students  were  given  a  72  item  questionnaire  in  which  11 
items  included  responses  that  might  suggest  alienation. 
The  authors  provide  an  analysis  of  the  results  of  each  of 
the  11  items  which  shows  that  various  degrees  of  alienation 
exist  in  every  school  included  in  the  sample.  Finally, 
•f-jno  nn+Knvo  antfcrpc-t-  imnl inations  of  these  findings  to 
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3)  Mirsky,  Joseph.   "Pupil-Event  Analysis  for  Needs  Assess- 

ment." Philadelphia:  Research  for  Better  Schools, 
December,  1969. 

The  author  presents  a  brief  argument  for  sampling  "pupil- 
events"  in  order  to  learn  about  the  nature  of  pupil-needs. 
He  suggests  that  pupil-need  assessments  examine  the  func- 
tions of  the  school  system  as  they  relate  to  the  things 
students  do. 

4)  Popham,  James  W.  "Educational  Needs  Assessment  in  the 

Cognitive,  Affective,  and  Psychomotor  Domain."  A 
presentation  at  three  ESEA  Title  III  Regional  Work- 
shops sponsored  by  the  U.S.  Office  of  Education, 
Ft.  Lauderdale,  Fla.,  November,  1969;  Philadelphia, 
Penna.,  December,  1969;  Scottsdale,  Ariz.,  December,  1969. 

A  first  problem,  identifying  educational  preferences,  can 
be  remedied  by  utilizing  the  behavioral  objectives  which 
are  being  developed  by  the  Instructional  Objectives  Exchange 
of  U.C.L.A.  and  then  sampling  the  clientele  to  determine 
the  priority  of  the  objectives.  A  second  problem,  the 
identification  of  the  learner's  status,  i.e.,  level  of 
attainment  can  be  solved  by  using  criterion-referenced 
tests  based  upon  the  behavioral  objectives.  The  third 
problem  involves  a  contrast  of  preferences  with  status, 
i.e.,  a  comparison  of  desired  outcomes  to  current  learner 
status,  can  be  solved  through  the  administration  of 
criterion  referenced  tests. 

(See  also  the  annotation  of  Marvin  Alkin's  article,  "Behavior 
Objective  Specifications  in  Evaluation:  Relevant  or 
Irrelevant"  which  appears  in  section  1.4). 

5)  Rhea,  Buford,  et  al.  "Measures  of  Child  Involvement  and 
Alienation  from  the  School  Program."  Chestnut  Hill, 
Mass.:  Boston  College,  1966. 

The  purpose  of  this  report  is  to  assess  the  alienation  and/ 
or  involvement  of  high  school  students  through  the  use  of 
short  interviews  supplemented  by  questionnaires.  The 
sample  consisted  of  two  highly-rated,  suburban  Boston 
schools  whose  students  were  of  middle  to  upper-middle 
class  backgrounds.  Although  the  authors  concluded  that 
students  did  not  display  the  traditional  symptoms  of  aliena- 
tion, they  were  not  able  to  evaluate  a  disturbing  anomaly 
which  resulted  in  students  converting  intended  educational 
goals  into  spurious  means  (good  grades,  admission  to  college, 
notion  of  "preparation").  They  feel  that  more  research 
should  be  conducted  to  evaluate  how  a  "task-orientation" 
view  of  schools  affects  student  alienation.   Furthermore, 
the  authors  characterize  a  pattern  that  has  developed  in  the 
two  schools  as  "institutional  paternalism."   In  the  final 
sections  of  the  report,  the  writers  discuss  some  of  the  con- 
ceptual problems  associated  with  measures  of  alienation  and 
involvement. 
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6)  Woodbury,  Charles  A.,  et  al.  "Research  Model  for  State  Edu- 
cational Needs  Assessment."  Paper  presented  at  the  Annual 
Meeting  of  the  American  Educational  Research  Association, 
Minneapolis,  March,  1970. 

This  paper  summarizes  the  Virginia  Needs  Assessment  Project's 
research  strategy  in  determining  learner-oriented  needs  in 
the  cognitive,  affective  and  psycho-motor  domains.  This 
approach  requires  the  delineation  of  goals  (four  are  mentioned), 
programmatic  efforts  (defined  as  implemented  goals),  and 
programmatic  outcomes  (measured  by  scores  on  standardized  tests 
and  scores  on  affective  rating  scales).  Needs  are  represented 
as  gaps  that  exist  between  the  goals  and  the  outcomes.  Also 
included  in  the  report  are  subordinate  assessment  strategies 
concerning  the  affective  domain  and  goal  perceptions. 

1.2  Overall  Object ives 

1)  A  Plan  for  Evaluating  the  Quality  of  Educational  Programs  in 

Pennsylvania:  Volumes  I,  II,  and  III.  Princeton,  N.J. : 
Educational  Testing  Service,  June,  1965. 

Initially,  the  report  lists  and  discusses  ten  educational  goals 
for  Pennsylvania  schools.   Included  is  a  proposed  survey  of 
public  opinion  to  assess  the  relative  importance  of  these  goals. 
A  methodology  is  outlined  for  the  development  of  performance 
indicators  which  eventually  would  result  in  inter- school  com- 
parisons. These  evaluations  would  point  to  needed  educational 
policy  adjustments  for  schools  with  comparable  inputs  and 
surround  ing  cond  itions . 

Note:  The  measures  designed  as  a  result  of  this  study  have  been 
used  in  Pennsylvania  by  the  Department  of  Education  in  what 
has  been  referred  to  as  "The  Quality  of  Education  Study." 

This  annotation  also  appears  in  section  1.4. 

2)  Downey,  Lawrence  W.   "The  Task  of  Public  Education."  Chicago: 

University  of  Chicago,  Midwest  Administration  Center, 
April,  1960. 

This  detailed  study  was  performed  to  determine  the  tasks  of 
public  education  (elementary  and  high  school)  and  the  import- 
ance of  these  tasks  (priorities)  as  seen  by  a  representative 
sample  of  educators  and  non-educators.  A  conceptual  framework 
outlining  the  tasks  of  public  education  was  formulated  from  an 
historical  overview  and  a  synthesis  of  ideas.  The  study 
utilized  the  Q-sort  technique  of  forced  choice  for  determining 
goal  priorities  of  schools. 
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3)  Educational  Goals  and  Objectives  in  California.  Sacramento 

Calif. :  California  School  Boards  Association,  August, 
1969. 

This  report  consists  of  the  goals  and  objectives  formulated 
by  900  school  districts  of  California  in  conjunction  with 
the  California  School  Boards  Association.  Distinctions  among 
philosophies,  goals,  and  objectives  are  described,  and  18 
general  goal  categories  have  been  generated.  Also,  51  entries 
appear  in  a  bibliography  on  goals  and  objectives. 

This  annotation  also  appears  in  section  5.0. 

4)  Goals  for  Public  Education  in  Texas.  A  Report  by  the  Sub- 

committee on  Goals  to  the  Governor's  Committee  on  Public 
School  Education,  U.S.  Office  of  Education.  Texas: 
Operation  PEP,  December,  1968. 

This  report  traces  the  historical  evolution  of  educational 
goals.   It  is  intended  as  an  aid  to  those  educational  planners 
and  managers  who  must  translate  educational  objectives  into^ 
operational  terms.  A  bibliography  of  consulted  references  is 
attached  to  the  study  as  well  as  appendixes  which  contain 
goal  statements  from  quoted  sources. 

This  annotation  also  appears  in  section  5.0. 

5)  Instructional  Goals.  Radnor,  Penna.:  Radnor  Schools, 

February,  1970. 

The  goals  developed  in  this  paper  are  modeled  after  the  goals 
generated  by  the  Educational  Testing  Service  for  the  Pennsyl- 
vania State  Board  of  Education.  The  goals  are  oriented  to 
specific  programs  within  these  content  categories  of  communi- 
cations, liberal  arts,  and  personal  development.  A  fourth 
goal,  organization,  involves  strategies  designed  to  present 
content  goals  most  effectively. 

1.3  Assignment  of  Weights  to  Objectives  and  Outcomes 

1)  Ackoff,  Russell  L.   "Towards  Quantitative  Evaluation  of  Urban 
Services,"  Public  Expenditure  Decisions  in  the  Urban 
Community,  (Washington:  Resources  for  the  Future,  Inc., 
1962). 

Ackoff  proposes  a  method  for  evaluating  community  values  asso- 
ciated with  a  collective  good  or  service.   He  lists  six  require- 
ments or  attributes  of  an  ideal  measure  of  performance  for 
determining  social  utility.   In  addition,  he  applies  his  evalua- 
tive technique  to  conceptual  problems  in  transportation  and 
education.   In  the  latter  regard,  Ackoff  sees  education  as  an 
accelerator  of  societal  progression  and  as  a  decelerator  of 
regression,  and  he  develops  a  theoretical  model  for  the  evalua- 
tion of  education  based  upon  this  progression-regression  prin- 
piple. 
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2)  Alesch,  D.  J.   Improving  Decision  Making  About  Priorities  in 

State  Government.   ClearinghouseTor  Federal  Scientific  and 
Technical  Information  Contract  &AD693648.   Santa  Monica, 
Calif.:  Rand  Corporation,  September,  1969. 

The  author  proposes  a  means  of  priority-setting  which  necessi- 
tates changes  in  the  institutional  framework  of  state  govern- 
ment. Alesch  stresses  the  need  for  a  procedure,  which  he 
refers  to  as  the  problem- opportunity  approach,  that  cuts 
across  traditional  groupings  of  public  agencies.  He  lists 
and  describes  four  criteria  for  determining  problem  priorities. 
Moreover,  he  enumerates  four  requirements  for  moving  towards 
a  system  of  priority-setting:  a  synthesis  of  planning  and 
budgeting  functions,  the  creation  of  needed  information 
systems,  the  need  for  participatory  priority-setting,  and  the 
necessity  of  a  flexible  system. 

3)  Anderson,  Donald  P.  "Clarifying  and  Setting  Objectives  on  an 

Intermediate  School  Districts  Objectives  Utilizing  the 
Delphi  Technique."  Paper  presented  at  the  Annual  Meeting 
of  the  American  Educational  Research  Association  Symposium 
on  Exploring  the  Potential  of  the  Delphi  Techniques  by 
Analyzing  its  Applications,  March,  1970. 

One  of  the  most  important,  but  often  neglected,  processes  in 
any  organization  is  that  of  explicating  and  setting  priorities 
on  objectives  or  target  conditions.  The  situation  is  con- 
founded in  service  systems  such  as  intermediate  school  organi- 
zations.  Consensus  on  such  priorities  is  rarely  achieved  or 
even  attempted.  The  Delphi  Technique,  built  on  the  strength 
of  informed  intuitive  judgement  and  designed  to  produce 
consensus,  provides  a  procedure  to  alleviate  this  problem 
situation. 

Experts,  identified  from  inside  and  outside  a  County  School 
District,  generated  an  initial  statement  of  objectives  by 
completing  a  stem  sentence  related  to  long-range  resource 
allocation.  The  statements  clustered  into  two  sets:  client 
services  and  organizational  adaptations.   Using  three  Delphi 
questionnaires,  priorities  were  assigned  to  each  set  independ- 
ently using  "zero  sum"  logic.  The  client  service  objectives 
are  being  used  currently  in  program  planning  efforts;  the 
organizational  adaptation  set  was  used  to  modify  the  County 
Office  organization. 

4)  Baumol,  William  J.   "Neumann-Morgenstern  Cardinal  Utility," 

Economic  Theory  and  Operations  Analysis,  (Englewood  Cliffs, 
W7JT:     Prentice-Hall,  Inc.,  1961),  pp.  331-346 . 

In  Chapter  17  the  author  presents  a  classic  method,  emerging 
from  economic  cardinal  utility  analysis,  of  assigning  utility  or 
worth  to  predictable  outcomes.   Baumol  mentions  that  a  calcula- 
tion based  on  the  utilities  of  alternative  outcomes,  rather  than 
the  magnitudes  of  outcomes,  may  be  more  useful  to  decision-makers. 
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5)  Churchman,  C.  West,  Russell  L.  Ackoff,  and  E.  Leonard  Arnoff, 

"Weighting  Objectives,"  Introduction  to  Operations 
Research,  (New  York:  John  Wiley  &  Sons,  Inc.,  1957), 
pp.  136-154. 

In  Chapter  7,  a  method  is  presented  for  estimating  the  relative 
value  of  objectives.   Initially,  the  authors  assume  that 
judgments  are  made  by  one  evaluator,  but  this  precondition  is 
relaxed  in  a  later  discussion  of  group  values.   Their  technique 
consists  of  a  "systematic  check  on  relative  judgments  by  a 
process  of  successive  comparisons."  An  easy-to-follow  outline 
both  for  a  small  and  a  large  number  of  objectives  is  provided 
along  with  two  illustrative  examples  for  further  understanding. 

6)  Dalkey,  N.   "Analyses  From  a  Group  Opinion  Study,"  Futures, 

I  (December,  1969),  pp.  541-551. 

The  author  conducted  various  experiments  in  the  use  of  a  Delphi 
questionnaire.   He  experimented  with  the  idea  that  point 
estimates  were  less  accurate  than  interval  estimates.  He 
tested  various  hypotheses  including  learning  as  a  function  of 
feedback,  time  as  related  to  the  accuracy  of  responses,  and 
overall  accuracy  as  a  function  of  confident  self-groups. 

Furthermore,  the  author  concludes  that  even  though  the  Delphi 
method  has  dealt  only  with  factual  judgments,  it  may  be 
applicable  to  the  weighting  of  value  judgments  (organization 
objectives).   If  value  judgments  are  indeed  "correct,"  then 
there  is  the  need  for  tests  of  individual  stability,  group 
convergence  and  group  reliability. 

7)  Downey,  Lawrence  W.  "The  Task  of  Public  Education."  Chicago: 

University  of  Chicago,  Midwest  Administration  Center, 
April,  1960. 

The  purpose  of  this  study  was  to  determine  the  tasks  of  public 
education  (elementary  and  high  school)  and  the  importance  of 
these  tasks  (priorities)  as  seen  by  a  representative  sample  of 
educators  and  non-educators.  A  conceptual  framework  outlining 
the  tasks  of  public  education  was  formulated  from  an  historical 
overview  and  a  synthesis  of  ideas.  The  study  utilized  the 
Q-sort  technique  of  forced  choice  for  determining  goal  priorities 
of  schools. 

Findings  are  presented  in  detail. 

This  annotation  also  appears  in  section  1.2. 
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Fallorl,  Carlos.  "Practical  Use  of  Decision  Theory  in  Value 

Engineering,"  Journal  of  Value  Engineering,  II  (May,  1954), 
pp.  45-49. 

Fallon  provides  a  brief  history  of  value  theory  from  Daniel 
Bernoulli  to  von  Neumann  and  Morgenstern,  Savage,  Raiffa  and 
Schlaiffer. 

Of  particular  interest  is  a  "selection  matrix"  which  combines 

estimate  of  effectiveness 


an  assignment  of  importance  with  an 
to  arrive  at  the  "value  of  the  i 


course  of  action." 

In  order  to  maintain  the  integrity  of  Fallon's  contribution  a 
crucial  paragraph  is  cited: 

"It  should  be  noted  that  we  do  not  use  the  word  effective- 
ness to  denote  weighted  efficiency,  as  is  sometimes  done 
in  a  game -theoretic  context.  For  our  purposes,  we  must 
accept  the  word  efficiency  in  the  ordinary  sense  of _ 
productivity  without  waste,  as  measured  by  a  comparison 
of  production  with  cost,  and  we  must  use  effectiveness 
also  in  its  ordinary  sense  of  producing  a  desired  effect 
(independent  of  the  cost  of  producing  such  effect)." 
(Author) 

9)  Fishburn,  Peter  C.  "Utility  Theory,"  Management  Science,  XIV 
(January,  1968),  pp.  335-378. 

"Utility  theory  is  interested  in  people's  preferences  or  values 
and  with  assumptions  about  a  person's  preferences  that  enable 
them  to  be  represented  in  numerically  useful  ways.  The  first 
two  sections  of  this  paper  say  more  about  what  utility  is, 
why  people  are  interested  in  it,  and  how  it  is  interpreted  and 
used  in  the  management  and  behavioral  sciences.  The  third 
section  summarizes  a  number  of  utility  theories:   it  may  be 
used  either  as  a  concluding  overview  of  the  range  and  variety 
of  utility  theories  or  as  a  bridge  to  the  final  sections. 
The  final  eight  sections  comprise  a  semi-technical  survey  of 
particular  theories  for  readers  interested  in  greater  depth.' 
(Author) 

(See  also  citation  in  section  5.0). 

10)  Grayson,  Jackson  C.   Decisions  Under  Uncertainty: Drilling 

Decisions  by  Oil  and  Gas  Operators.   Boston:  Harvard 
Business  School,  1960. 

Grayson's  study  of  decision-making  in  the  oil  drilling  industry 
is  a  classic  if  only  for  irs  clear  presentation.  This  analysis 
of  decisions,  using  Bayesian  statistics,  is  one  of  the  first 
applied  efforts  found  in  the  literature.   In  addition,  Grayson 
does  some  of  the  first  experimental  empirical  work  on  individual 
utility  functions. 
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11)  Helmer,  Olaf.  "Convergence  of  Expert  Consensus  Through  Feed- 

back." Santa  Monica,  Calif.:  Rand  Corporation, 
September,  1964. 

This  is  one  of  the  earliest  reports  in  the  literature  of  an 
application  of  the  Delphi  Method.  An  effort  is  made  to 
explore  some  assumptions  implicit  in  the  use  of  this  method. 

12)  Huber,  George  P.,  Vinod  K.  Sahney,  and  David  L.  Ford.   "A 

Study  of  Subjective  Evaluation  Models,"  Behavioral  Science, 
XIV  (1969),  pp.  483-489. 

"The  research  reported  explores  some  questions  which  arise 
when  staff  groups,  such  as  operations  research  groups,  are 
faced  with  obtaining  a  quantitative  representation  of  an 
individual's  or  an  organization's  evaluation  scheme.  The 
study  is  concerned  with  the  ability  of  professional  hospital 
personnel  to  develop  and  use  a  subjective  evaluation  model  in 
a  reliable  manner,  the  possible  forms  of  such  a  model,  and 
the  extent  to  which  the  models  used  by  the  professionals  re- 
flected some  recognizable  organizational  evaluation  scheme." 
(Author) 

13)  Klahr,  David.  "Decision  Making  in  a  Complex  Environment:  The 

Use  of  Similarity  Judgments  to  Predict  Preferences," 
Management  Science,  XV  (July,  1969),  pp.  595-618. 

"Judgments  of  the  relative  similarity  of  pairs  of  alternatives 
are  used  to  construct  a  model  of  the  decision  space  of  a  group 
of  college  admissions  officers.  This  model  is  then  used  to 
predict  the  preferences  of  the  officers.  The  accuracy  of  the 
predictions  supports  the  hypothesis  that  preference  judgments 
are  made  on  the  basis  of  the  similarity  of  given  alternatives 
to  an  "ideal"  alternative.  A  nonmetric  multidimensional 
scaling  procedure  is  used  to  construct  the  space.   This  pro- 
cedure yields  a  dimensional  representation  based  upon  very 
few  assumptions  about  the  nature  of  the  similarity  measures." 
(Author) 

14)  Peper,  John  B.   "Summary  of  Program  Relevance  Evaluation  Model.' 

An  address  prepared  for  the  American  Educational  Research 
Association,  Minneapolis,  March,  1970. 

This  paper  may  be  viewed  as  a  companion  to  the  paper  by  Ying 
Chuang  and  Peper.   (See  section  2.1)  The  major  purpose  of 
Peper' s  paper  is  to  report  the  development  and  use  of  the  model 
in  Philadelphia  rather  than  a  detailed  description  of  the  model 
itself.  The  discussion  treats  the  assignment  of  priorities 
in  the  form  of  relative  weights  to  projects.  Peper  concludes 
that  the  model  appears  to  be  useful  in  program  planning  and  in 
making  new  funding  decisions. 

This  annotation  also  appears  in  section  2.4. 
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15)  Reisman,  Arnold,  and  Martin  I.  Taft.  "A  Systems  Approach  to 

the  Evaluation  and  Budgeting  of  Educational  Programs," 
Socio-Economic  Planning  Sciences,  III  (1969),  pp.  245-277. 

This  item  refers  specifically  to  pages  247-251,  whose  explicit 
purpose  is  to  develop  a  composite  utility  function  for  learning 
effectiveness  so  that  resources  can  be  allocated  in  order  to 
maximize  that  function.  The  authors  enumerate  and  discuss 
their  utility  criterion.  They  develop  a  theoretical  approach 
which  measures  the  relative  utility  of  competing  departments 
in  relation  to  total  utility,  given  an  environment. 

16)  Rescher,  Nicholas.  "Delphi  and  Values."  Santa  Monica,  Calif.: 

Rand  Corporation,  September,  1969. 

The  author's  primary  aim  is  to  provide  a  discussion  of  theoreti- 
cal background  considerations  for  the  application  of  the 
Delphi  method.   Seven  aspects  are  considered  on  pages  16  and 
17.  Among  these  are  operative  values,  value  criteriology , 
divergences  of  group  value  posture,  value  consensus,  and  third 
party  evaluation  of  conflicts  of  interest. 

17)  Rothenberg,  Jerome.  The  Measurement  of  Social  Welfare.  Engle- 

wood  Cliffs,  N.J.:   Prentice-Hall,  Inc.,  1961. 

Rothenberg  surveys,  in  depth,  a  substantial  portion  of  welfare 
economics  theory.  This  book  treats  many  topics  among  which 
are  sections  on  preferences,  utilities,  and  social  welfare 
functions. 

18)  Schimpeler,  Charles  C,  and  William  L.  Greece  "Systems 

Evaluation:  An  Approach  on  Community  Structure  and  Values," 
Highway  Research  Record,  CCXXXVIII  (1968). 

In  the  early  sections  of  their  report,  the  authors  summarize 
various  efforts  in  systems  evaluation.  Their  analytical  method 
of  plan  evaluation  has  three  stages.  First,  in  a  Louisville 
study,  the  authors  describe  their  technique  for  obtaining  a 
set  of  weighted  objectives.  Second,  they  present  their 
evaluation  model  which  consists  of  an  effectiveness  matrix  and 
a  utility-based  decision  model.  Third,  they  introduce  a 
modification  which  takes  into  account  the  effects  of  socio- 
economic strata  in  the  community. 
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19)  Stimson,  David  H.  "Utility  Measurement  in  Public  Health 

Decision  Making,"  Management  Science,  XVI  (October, 
1969),  pp.  B-17-B-30. 

"The  Churchman-Ackoff  approximate  measure  of  value  method 
was  used  successfully  to  measure  the  utilities  of  objectives 
of  decision  makers  in  a  large  public  health  agency.  The 
utilities  thus  obtained  formed  part  of  a  model  of  a  resource 
allocation  problem  faced  by  the  agency  in  its  role  as 
allocator  of  a  federal  grant.  A  comparison  of  the  normative 
solution  derived  from  the  model  with  the  solution  already  ^ 
decided  upon  by  the  agency  tended  to  support  the  hypothesis^ 
that  the  members  of  the  agency  chose  among  alternatives  as  if 
they  were  maximizing  expected  utility."  (Author) 

(See  Churchman,  Ackoff  and  Arnoff  citation  in  section  1.3. 
See  also  citation  in  section  5.0.) 

20)  Torgerson,  Warren  S.  Theory  and  Methods  of  Scaling.  New 

York:  John  Wiley  &  Sons,  Inc.,  1958. 

This  is  a  reasonably  up-to-date  book  about  scaling  theory. 
One  may  draw  two  conclusions:  scaling- theory  and  methods 
are  cumbersome,  and  scaling- theory  and  methods  have  little 
to  contribute  to  the  solution  of  problems  of  assigning  weights 
to  outcomes  by  single  and  group  decision-makers. 

21)  Winkler,  Robert  L.   "The  Quantification  of  Judgment:   Some 

Methodological  Suggestions,"  Journal  of  the  American 
Statistical  Association,  LXII  (December,  1967),  pp.  1105- 
1120. 

"The  personalistic  theory  of  probability  prescribes  that  a 
person  should  use  personal  probability  assessments  in  decision- 
making and  that  these  assessments  should  correspond  with  his 
judgments.   Since  the  judgments  exist  solely  in  the  assessor's 
mind,  there  is  no  way  to  prove  whether  or  not  this  requirement 
is  satisfied.  De  Finetti  has  proposed  the  development  of 
methods  which  should  oblige  the  assessor  to  make  his  assess- 
ments correspond  with  his  judgments.  An  ideal  assessor  is 
hypothesized  and  his  behavior  is  investigated  under  a  number 
of  such  methods  (including  those  suggested  by  de  Finetti  and 
others).  The  implications  of  these  methods  for  the  theory  of 
personal  probability  are  discussed.  Finally,  although  the 
present  interest  is  primarily  normative,  the  practicability  of 
the  methods  is  also  discussed."  (Author) 

Winkler's  analysis  pertaining  to  aspects  of  subjective  or 
personal  probabilities  seems  to  have  a  great  deal  of  relevance 
and  carry-over  to  the  assignment  of  subjective  ratings.  A 
brief  bibliography  at  the  end  of  this  article  is  crammed  with 
references  to  theory  on  subjective  probabilities  and  Bayesian 
statistical  analysis. 
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1.4  Performance  Criteria 

1)  Alkin,  Marvin  C.  "Behavior  Objective  Specifications  in  Evalua- 

tion: Relevant  or  Irrelevant?"  Western  Reg innal  Conference 
on  Testing  Problems,  (Princeton,  N.J.:  Educational  Testing 
Service,  May,  1969),  pp.  1-13. 

The  purpose  of  this  paper  is  to  determine  when  behavioral 
objectives  are  helpful  in  the  process  of  evaluation.  The  autnor 
defines  evaluation  and  lists  its  five  stages:  systems  assess- 
ment (determination  of  needs  and  objectives),  planning  (selec- 
tion of  a  program),  program  implementation  (assessment  of  actual 
versus  intended  effects),  program  improvement  (presentation 
of  interim  feedback  to  the  decision-maker),  and  program  certifi- 
cation (presentation  of  final  feedback  to  the  decision-maker ). 
Behavioral  objectives  are  useful  in  the  first,  fourth  and  tircn 
stages.  Although  there  is  little  evidence  to  suggest  that 
decisions  are  affected  by  the  presentation  of  behavioral  ob- 
jectives, Alkin  has  established  an  Instructional  Objectives 
Exchange  at  UCLA  to  provide  local  decision-makers  with  behav- 
ioral objectives  and  test  items. 

2)  A  Plan  for  Evaluating  the  Quality  of  Educational  Programs  in 

Pennsylvania:  Volumes  I,  II,  and  III.   Princeton,  N.J.: 
Educational  Testing  Service,  June,  1965. 

Initially,  the  report  lists  and  discusses  ten  educational  goals 
for  Pennsylvania  schools.   Included  is  a  proposed  survey  ot 
public  opinion  to  assess  the  relative  importance  of  these 
goals.  A  methodology  is  outlined  for  the  development  ot  per- 
formance indicators  which  eventually  would  result  in  inter 
school  comparisons.  These  evaluations  would  point  to  needed 
educational  policy  adjustments  for  schools  with  comparable 
inputs  and  surrounding  conditions. 

Note:  The  measures  designed  as  a  result  of  this  study  have 
been  used  in  Pennsylvania  by  the  Department  of  Education  in 
what  has  been  referred  to  as  "The  Quality  of  Education  Study. 

This  annotation  also  appears  in  section  1.2. 

3)  Brink,  Charles  B.   Social  Change  Evaluation  Project  Final 

Report  Volume  I.   Clearinghouse  tor  Federal  Scientific  and 
Technical  Information  Contract  #184522,  Washington,  D.C. : 
Washington  University,  June,  1968. 

Pages  80-83  are  devoted  to  the  question  of  effectiveness.  The 
author  lists  three  possible  ways  to  measure  effectiveness: 
measurement  of  the  impact  in  terms  of  organizational  objectives, 
measure  the  difference  as  a  consequence  of  the  program  insti- 
tuted, and,  establishment  of  a  set  of  exploratory  probes  to 
measure  selected  points  of  impact.  The  authors  describe  a 
number  of  difficulties  in  using  either  the  first  or  second 
approaches.  Characteristics  and  examples  of  exploratory  probes 
are  discussed. 
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4)  Dyer,  Henry  S.  "The  Concept  and  Utility  of  Educational  Per- 

formance Indicators."  Paper  presented  at  the  Systems 
Science  and  Cybernetics  Conference,  Boston,  October, 
1967. 

The  author  describes  a  technique  for  the  development  of 
educational  performance  indicators  based  upon  regression 
analysis.  A  theoretical  model  is  presented  to  the  reader 
with  suggestions  and  interpretations  of  its  use.  Possible 
problems  of  implementation  are  discussed.  Finally,  Dyer 
implies  that  educational  performance  indicators  may  "provide 
a  basis  for  genuine  cost-benefit  analysis." 

This  annotation  also  appears  in  section  2.5. 

5)  Eiss,  Albert  F.,  and  Mary  Blatt  Harbeck.  Behavioral  Objectives 

in  the  Affective  Domain.  Washington,  D.C.:  National 
Science  Supervisors  Association,  1969. 

The  primary  aim  of  this  monograph  is  to  present  a  methodology 
for  developing  behavioral  objectives  in  the  affective  domain 
of  science  in  order  to  create  "scientific  literacy"  in  students. 
The  report  focuses  on  the  difficulty  of  assessing  whether  or 
not  an  overt  student  behavior  is  indicative  of  the  desired 
objective.  This  slippage  (which  the  authors  refer  to  as  the 
"credibility  gap")  is  especially  apparent  in  the  affective 
domain.  Therefore,  the  authors  suggest  the  use  of  "open-loop" 
behavioral  objectives,  rather  than  the  "closed-loop"  types 
which  are  limited  to  the  cognitive  area.   In  addition,  the 
appendices  provide  examples  of  affective  behavioral  objectives, 
evaluation  items  in  the  affective  domain,  and  evaluation 
instruments  suitable  to  this  area. 

6)  Friedly,  Philip  H.,  et  al.  "Benefit-Cost  Applications  in  Urban 

Renewal:  A  Feasibility  Study."  Clearinghouse  for  Federal 
Scientific  and  Technical  Information  Contract  #182969, 
Bethesda,  Md . :  Resource  Management  Corporation,  August, 
1968. 

This  item  refers  to  Chapter  6,  of  this  report,  "Evaluating  the 
Impacts  of  Renewal  Activities."  A  matrix  is  developed  showing 
the  interactions  between  11  urban  renewal  objectives  and  11 
welfare  indicators.   Measuring  these  11  welfare  indicators 
would  yield  information  regarding  the  attainment  of  the 
specified  objectives.  Nevertheless,  in  their  discussion  of 
each  of  the  welfare  indicators,  the  authors  point  out  that  the 
criteria  are  not  necessarily  commensurable.   Empirical  testing 
must  now  provide  information  about  usefulness  and  administrative 
costs  for  the  respective  indicators. 

This  annotation  also  appears  in  section  3.1. 


-31-    CPL  Exchange  Bibliography  #219 

7)  Ha try,  Harry  P.  Criteria  for  Evaluation  in  Planning  State 

and  Local  Programs.   A  Study  Submitted  by  the  Sub- 
committee on  Intergovernmental  Relations  to  the  Committee 
on  Government  Operations.  Washington,  D.C. :  George 
Washington  University,  July,  1967. 

This  paper  identifies  specific  criteria  for  use  in  evaluating 
alternative  plan  proposals  in  seven  major  program  areas  within 
a  PPBS  format.  This  work  is  particularly  adaptable  to  state 
and  local  government  functions.  The  author  describes  the 
properties  of  ideal  criteria  and,  moreover,  he  lists  and  dis- 
cusses 17  other  considerations  and  attributes  of  his  criteria. 
Most  of  those  enumerated  are  nonmonetary  criteria.  In  addition, 
Hatry  indicates  where  his  performance  measures  may  be  deficient 
and  also  where  they  may  overlap  into  other  program  categories. 
For  the  reader's  reference,  a  PPBS  government  program  structure 
is  illustrated  in  the  appendix. 

8)  Kurland,  Norman  D.  "Developing  Indicators  of  Educational 

Performance,"  Educational  Technology,  VII  (1967). 

The  author  identifies  some  of  the  fundamental  aspects  of 
educational  performance  indicators.  First,  he  lists  some 
reasons  why  educational  performance  has  been  difficult  to 
assess,  then  he  proceeds  to  specify  the  attributes  of  good 
performance  measures.   In  addition,  he  delves  into  the  problems 
of  data  collection,  input  factor  measurement,  process  factor 
quantification,  and  output  factor  adequacy.   In  the  latter 
regard,  he  gives  special  mention  to  Ralph  Tyler's  national 
assessment  project.  Finally,  he  suggests  that  an  adaptation 
of  econometric  models  may  have  some  applicability  towards 
developing  educational  performance  indicators. 

9)  McDevitt,  Matthew,  and  Thomson  McGowan.   "New  York  State's 

Central  Social  Environment  Study."  Albany,  N.Y.:  New 
York  State  Office  of  Planning  Coordination,  January, 
1970. 

"This  monograph  describes  the  background  and  current  status 
of  a  Central  Social  Environment  Study  which  the  New  York 
State  Office  of  Planning  Coordination  is  designing.  The 
report  describes  the  need  for  such  a  monitoring  system  to 
provide  the  state's  decision  makers  with  continuous  informa- 
tion  on  social  attitudes  and  problems  and  to  evaluate  systemati- 
cally the  state  programs  which  deal  with  these  problems,  and 
steps  being  taken  to  meet  this  need."  (Author) 
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10)  Miller,  James  R.  Assessing  Alternative  Transportation  Systems. 

Clearinghouse  for  Federal  Scientific  and  Technical  Informa- 
tion Contract  #185167.   Santa  Monica,  Calif.:  Rand 
Corporation,  April,  1969. 

This  item  is  specifically  concerned  with  sections  of  the  report 
dealing  with  the  concept  and  measure  of  "worth" .  Miller  de- 
fines the  "worth"  of  an  alternative  as  its  ability  to  accomplish 
a  given  objective.  Worth  does  not  require  a  consensus  among 
decision-makers  nor  does  it  specify  temporal  stability.  The 
author  describes  the  essential  features  of  his  scale  of  worth. 
For  example,  his  scale  ranges  from  +1  to  -1  where  positive 
values  imply  positive  worth  points  (gratiles),  negative  values 
suggest  negative  gratification  points  and  "0"  represents  a 
point  of  indifference.  Assigned  worth  numbers  are  assumed  to 
occur  with  complete  certainty,  therefore  a  risk  factor  is  not 
included  in  Miller's  formulation.   (It  should  be  pointed  out 
that  Miller's  concept  of  "worth"  is  consistent  with  concepts 
of  "performance"). 

11)  Mood,  Alex  M.   "Macro-Analysis  of  the  American  Educational 

System,"  Operations  Research,  XVII  (September-October, 
1969),  pp.  770-784. 

"This  paper  presents  a  rudimentary  model  of  the  public  school 
system  as  an  input-output  process.  The  inputs  are  students' 
own  abilities  and  attitudes,  parental  support,  peer  support, 
quality  of  the  school  system,  community  support,  and  society's 
posture  with  respect  to  education.  Outputs  are  various  cate- 
gories of  academic  achievement  as  well  as  social  competence, 
responsibility,  self  confidence,  creativeness,  ethics,  and 
ambition.  All  these  factors  must  be  measured  by  index  numbers 
or  simple  indicators.  The  model  is  a  set  of  regression  equations 
relating  outputs  to  inputs.   Some  implementation  of  the  model 
has  been  made  possible  by  means  of  data  gathered  in  the  U.S. 
Office  of  Education's  Equality-of-Educational-Opportunity 
Survey."  (Author) 

This  annotation  also  appears  in  section  2.5. 

12)  Richard,  Robert.   Subjective  Social  Indicators.   Clearinghouse 

for  Federal  Scientific  and  Technical  Information  Contract 
#PB187944.   Chicago:  National  Opinion  Research  Center, 
September,  1969. 

The  purpose  of  this  report  is  to  describe  the  development  of 
a  "social  profile  of  target  areas."  The  author  suggests  that 
this  report  could  be  useful  to  program  planners  in  locally 
sponsored  surveys.  The  report  covers  the  following  topics: 
a  review  of  the  literature  on  social  indicators  and  program 
planning  and  evaluation  research,  the  research  methodology 
employed,  a  discussion  of  survey  measurement  and  question 
design,  a  listing  of  the  set  of  items  that  encompass  the  social 
profile,  and  the  development  and  rationale  for  some  of  the 
items  used . 
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13)  Sisson,  Roger  L.,  and  Edwin  C.  Brewin.   "An  Introduction  to 

the  Education-Planning-Programming-Budgeting  System." 
Philadelphia:  Government  Studies  Center  of  the  Fels 
Institute,  University  of  Pennsylvania,  November,  1969. 

Sisson  describes  EPPBS  (the  "E"  stands  for  education)  as  a 
method  which  helps  the  school  district  determine  its  present 
situation  and  project  various  ways  in  which  the  school  dis- 
trict could  operate  in  the  future.  An  EPPBS  planning  metho- 
dology is  presented.  An  essential  element  of  EPPBS  is  the 
indicator.   Indicators  are  developed  for  both  inputs  and  out- 
puts under  conditions  of  no  over-all  policy  changes  (baseline 
projection)  and  under  policy  adjustment  (the  selection  of  a 
plan).  They  also  show  gaps  between  present  operations  and 
future  requirements.  These  gaps  become  objectives  from  which 
projects  are  generated  to  close  indicator  gaps.   In  sum,  EPPBS 
permits  the  decision-mPker  to  see  the  possible  consequences  of 
"plans  of  operation  in  terms  of  both  indicators  (which  show 
benefits  and  effects  derived)  and  financial  factors  (which 
measure  the  costs)." 

This  annotation  also  appears  in  section  2.2. 

14)  Webb,  Eugene  J.,  et  al.   Unobstrusive  Measures:   Nonreactive 

Research  in  the  Social  Sciences.   Chicago:  Rand  McNally 
&  Company,  1966. 

The  main  concern  of  the  authors  lies  in  an  over-reliance  on  the 
interview  and  questionnaire  as  means  of  data  collection.  They 
favor  a  multiple  measurement  scheme  which  would  utilize  com- 
plementary techniques  to  test  hypotheses  and  to  broaden  the 
perception  of  social  scientists.   First,  the  authors  explain 
some  of  the  limitations  of  traditional  methods  mentioned.  Then 
they  discuss  and  report  studies  that  have  used  supplementary, 
unobtrusive  measures.  Most  of  these  methods  are  free  from  the 
essential  bias  of  the  interview  and  questionnaire  --  the  re- 
spondent's reactive  effects.   Some  of  the  methods  recommended 
include  the  use  of  physical  traces,  archives  and  simple  observa- 
tion. The  authors  describe  the  advantages  as  well  as  the 
pitfalls  associated  with  each  technique. 

1.5  Cost  and  Accounting  Problems 

1)  A  Study  Design  for  a  Comprehensive  Planning  Program  in  the 

Denver  Region.   Clearinghouse  for  Federal  Scientific  and 
Technical  Information  Contract  #6808.   Denver:  Denver 
Regional  Council  of  Governments,  December,  1968. 

The  fourth  chapter  of  the  Denver  Regional  Council  of  Governments 
comprehensive  plan  discusses  the  development  of  a  system  for 
managing  the  studies  that  are  to  be  used  as  inputs  to  the 
overall  planning  process.   Included  are  descriptions  of  man- 
power and  cost  requirements,  programming  and  scheduling  of 
activities,  implications  with  respect  to  organization  and  the 
development  of  a  management  information  system. 


-34-    CPL  Exchange  Bibliography  #219 

2)  Kelly,  James  A.  "Budget  Decision  Making  in  Large  Public  School 

Systems,"  Socio-Economic  Planning  Sciences,  II  (1969), 
pp.  135-140. 

"This  paper  reviews  and  discusses  results  of  a  study  of  the 
budget  process  in  fourteen  large  city  school  districts. 
Topics  covered  include  incremental  decision  making,  the  use 
of  "ratios"  and  "norms"  in  school  budgeting,  public  partici- 
pation in  budgeting,  relevant  structural  arrangements  of  local 
government,  and  the  influence  of  boards  of  education  on  the 
allocation  of  resources.  Finally,  implications  for  the  improve- 
ment of  school  management  practices  in  large  school  districts 
are  discussed."  (Author) 

3)  King,  Barry  G.  "Cost-Effectiveness  Analysis:   Implications  for 

Accountants,"  Journal  of  Accountancy,  CXXIX  (March,  1970), 
pp.  43-49. 

This  article  suggests  that  accountants  should  become  more  aware 
of  the  uses  and  limitations  of  cost-effectiveness  analysis. 
Accountants  should  take  a  more  comprehensive  view  of  organi- 
zational objectives  and  costs.  The  author  feels  that  part  of 
the  accounting  function  is  to  assist  management  in  the  search 
for  measures  of  effectiveness. 

4)  Temkin,  Sanford.   "Cost-Effectiveness  Evaluation  as  an  Input 

into  the  Budgetary  Process."  Philadelphia:  Research  for 
Better  Schools,  Inc.,  June,  1969. 

The  author  presents  a  four-year  plan  suggesting  how  a  school 
district  could  make  a  transition  from  a  line  item  budget  to  one 
which  reflects  school  objectives  and  activities.  The  key  to 
implementing  such  a  system,  the  author  points  out,  is  to  allow 
more  people  in  the  school  system  to  participate  in  the  planning 
process. 

5)  Tenzer,  A.J.,  J.B.  Benton,  and  C.  Teng.  "Applying  the  Con- 

cepts of  Program  Budgeting  to  the  New  York  City  Police 
Department."  Santa  Monica,  Calif.:  Rand  Corporation, 
June,  1969. 

This  memorandum  describes,  in  a  very  pragmatic  manner,  three 
phases  of  the  PPB  System  developed  for  the  New  York  City  Police 
Department:  the  states  of  developing  a  program-budgeting  format 
for  five  essential  programs,  a  cost-estimating  methodology  for 
two  major  cost  categories,  and  an  on-line  computer  model  for 
estimating  cost  implications  of  alternative  resource  alloca- 
tions. 

This  annotation  also  appears  in  section  2.2. 
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2.0  SYSTEM  WIDE  PLANNING  METHODS 


2.1  Comprehensive  Planning 


1)  Chuang,  Ying  C,  and  John  B.  Peper.  "The  Philadelphia  Model 

for  Title  I  Evaluation."  Philadelphia:   School  District 
of  Philadelphia,  April,  1969. 

The  authors  describe  the  development  of  an  evaluation  model 
for  Title  I  projects  in  the  Philadelphia  School  System.  The 
evaluation  model  has  two  major  divisions:  an  individual 
project  evaluation  model,  and  an  overall  cost-effectiveness 
model.  The  latter  was  formulated  so  that  meaningful  comparisons 
could  be  made  among  projects.  The  authors  present  a  theoretical 
discussion  of  the  evaluation  model. 

This  annotation  also  appears  in  section  2.4. 

2)  Magoon,  Jon,  and  Francis  Tannian.  "School  System  Analysis: 

An  Interaction  Process  Model."  Newark,  Del.:  University 
of  Delaware,  March,  1970. 

"The  model  outlined  here  describes  sociological,  political, 
economic,  and  educational  interactions  in  an  urban  school 
system  in  order  to  describe  how  the  system  is  performing  and 
might  be  amenable  to  change  and  reform."  The  authors  describe 
their  model  as  a  multi-stage  input/output  scheme  which  draws 
some  of  its  technique  from  Jesse  Burkhead's  input/output  model 
for  city  school  systems.   Six  variables  have  been  chosen  for 
study:  pupil  achievement,  the  community,  the  bureaucracy, 
the  political  elites,  the  teacher,  and  economic  resource 
allocation.  Ultimately,  the  information  gathered  will  have 
practical  utility  for  the  school  program  decision-maker. 

3)  Sandberg,  John  (ed.).   Schools  and  Comprehensive  Urban  Planning. 

Portland,  Oreg. :  Northwest  Regional  Educational  Labora- 
tory, September,  1969. 

The  purpose  of  this  document  is  to  suggest  ideas  that  will  help 
school  administrators  work  more  effectively  with  local,  state 
and  regional  planning  bodies.  The  authors  stress  the  urgency 
for  comprehensive,  interagency  planning  in  urban  areas.  A 
series  of  articles  examines  some  aspects  of  integrative,  cooper- 
ative planning.   For  example,  John  Sandberg' s  article  assesses 
some  of  the  problems  that  beset  agencies  that  try  integrative 
planning.   In  an  article  entitled,  "Urban  Planning  Objectives," 
the  authors  list  certain  "do's  and  don'ts"  for  involving  citi- 
zens in  the  planning  process.  They  also  state  that  the  quality 
of  public  participation  depends  largely  on  the  peoples'  access 
to  facts  and  ideas.  The  latter  two  articles  by  Hansen  and 
Musolf  discuss  planning  models  (types  of  models  and  strategies 
used  to  bring  about  change)  and  the  stages  in  the  development 
of  an  information  system  for  urban  areas. 

This  annotation  also  appears  in  sections  3.2  and  4.1. 
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4)  Terakin,  Sanford.  "Comprehensive  Planning  for  School  Districts." 

Paper  prepared  for  American  Educational  Research  Associa- 
tion Symposium,  Minneapolis,  March,  1970. 

The  author  feels  that  current  "PPBS  approaches"  present  serious 
implementation  problems  for  school  districts.  He  provides  a 
methodology  for  engaging  in  comprehensive,  system-wide  plan- 
ning for  school  districts.  The  author  contends  that  an  evolu- 
tionary installation  of  the  planning  mechanism  is  required 
because  the  planning  mechanism  itself  causes  a  great  deal  of 
change.  Once  the  planning  mechanism  has  been  installed  informa- 
tion flows  can  be  generated  to  evaluate  the  system  as  it  is, 
in  order  to  form  a  basis  for  future  change  decisions. 

This  annotation  also  appears  in  section  4.2. 

5)  Temkin,  Sanford.  "A  Comprehensive  Theory  of  Cost- Effectiveness." 

Philadelphia:  Research  for  Better  Schools,  Inc.,  April, 
1970. 

The  author  notes  two  deficiences  that  stem  from  contemporary 
cost-effectiveness  methods.   Each  technique  suffers  from  a 
lack  of  a  firm  theoretical  base  of  assumptions,  and  an  absence 
of  a  means  for  selecting  an  efficiency  technique  for  a  practi- 
cal problem.  This  paper  is  intended  to  correct  these  inade- 
quacies and  to  appeal  to  the  mathematically- trained  person  who 
is  interested  in  a  general  theory  of  school  system  planning  and 
in  methods  such  as  cost/benefit  and  cost-effectiveness  analysis. 
Temkin  presents  a  theoretical  development  of  9  decision  cases 
in  an  evolutionary  framework  ranging  from  the  simplest  decision- 
making situation  (Case  1)  to  complex  circumstances  faced  by 
school  administrators  (Cases  8  and  9).  The  cases  are  augmented 
by  mathematical  proofs.   In  addition,  the  author  discusses  his 
rationale  for  recommending  the  use  of  cost-effectiveness  for 
school  district  comprehensive  planning  rather  than  benefit/ 
cost  analysis. 

This  annotation  also  appears  in  section  2.4. 
2.2  PPBS 

1)  Judy,  Richard  W.  "Systems  Analysis  for  Efficient  Resource 

Allocation  in  Higher  Education."  Canada:   University  of 
Toronto,  Institute  for  the  Quantitative  Analysis  of 
Social  and  Economic  Policy,  January,  1970. 

This  article  describes  the  major  projects  of  the  Research  Program 
on  Systems  Analysis  for  Efficient  Resource  Allocation  in  Higher 
Education  and  indicates  their  status  at  the  end  of  1969.  Men- 
tion is  made  of  the  PPBS  project  and  its  major  tasks,  a  sche- 
matic is  included  of  CAMPUS,  a  simulation  model,  and  attention 
is  devoted  to  the  objectives  and  status  of  integrated  university 
information  systems.  Space  is  also  given  to  project  facilities 
and  university  information  resource  center. 

(See  also  citation  in  section  5.0). 
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2)  McAbee,  Harold  V.  "Planning-Programming-Budgeting  Systems 

and  State  Educational  Agency  Administration."  Washing- 
ton, D.C.:  United  States  Office  of  Education  Department 
of  Health,  Education,  and  Welfare,  February,  1969. 

This  item  refers  to  a  position  paper  prepared  for  The  United 
States  Office  of  Education  Department  of  Health,  Education, 
and  Welfare  by  Harold  V.  McAbee  of  the  Teaching  Research 
Division  of  the  Oregon  State  System  of  Higher  Education.  The 
report  is  divided  into  three  sections.  Primarily,  a  summary 
of  the  attributes  and  design  of  PPBS  with  implications  for 
application  to  state  education  agencies  is  contained  in  Parts 
I  and  II.  Part  III  is  devoted  to  two  plans,  one  for  the  U.S. 
Office  of  Education  and  the  other  for  state  school  officers, 
in  order  to  field  test,  revise  and  implement  a  PPB  System. 

3)  McAbee,  Harold  V.  The  Oregon  Planning  Programming  Budgeting 

Systems  Institute.  Monmouth,  Oregon:  Teaching  Research 
Division,  Oregon  State  System  of  Higher  Education, 
December,  1968. 

A  report  of  an  Institute  [August  19-30,  1968^ designed  to 
acquaint  state  level,  vocational  education  administrators  to 
the  potential  and  structure  of  Planning,  Programming,  Budget- 
ing Systems,  a  Program/ Systems  approach  to  budgeting.  A 
variety  of  curriculum  materials  were  collected  and  developed. 
The  report  includes  substantial  content  about  PPBS.  The  need 
and  potential  for  a  new  approach  to  budgeting  are  outlined  as 
well  as  areas  for  further  research,  development,  and  applica- 
tion.  Briefs  of  all  Institute  presentations  and  other 
activities  are  included  along  with  extensive  bibliographical 
references,  Institute  program,  participants,  and  evaluation. 
Basic  economic  principles  which  undergird  PPBS  are  explained. 
Applications  of  cost/benefit  and  cost-effectiveness  analysis 
are  detailed.   (ERIC  Report  Resume) 

(See  also  citation  in  section  5.0). 
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4)  McGivney,  Joseph  H.,  and  William  C.  Nelson.  Program,  Planning, 

Budgeting  Systems  for  Educators,  Volumes  I,  II ?   III:  An 
Annotated  Bibliography.   Columbus,  Ohio:  Ohio  State 
University,  December,  1969.   ERICft  ED  032417  (Vol.  I), 
ERIC#  ED  032418  (Vol.  II),  ERIC  #  ED  035756  (Vol.  III). 

"Planning,  Programming,  Budgeting  System  (PPBS)  includes  con- 
cepts and  techniques  for  resource  allocation  decision  making 
for  rational  and  effective  programming.  Most  federal  agencies 
use  PPBS  for  the  development,  analysis  and  presentation  of 
resource  needs.  Traditional  methods  of  budgeting  focus  pri- 
marily on  resource  inputs.   PPBS  focuses  on  both  inputs  (cost) 
and  outputs  (benefits).  The  guidelines  in  this  volume  suggest 
initial  training  in  a  sequential  pattern  for  a  course  in  PPBS. 
Examples  used  are  drawn  from  the  field  of  vocational  education. 
Included  in  the  instructional  outline  are  sections  relating  to: 
(1)  Overview,   (2)  Systems  Theory,   (3)  The  Planning  Process, 
(4)   Investment  Alternatives,   (5)  Program  Budgeting, 
(6)  Analysis  of  Alternatives,   (7)  Programming  and  Manage- 
ment Control,  (8)  Basic  Data  for  PPBS,  and  (9)  Limitations 
of  PPBS.  The  document  contains  three  parts:  The  What-When- 
Where-Who-Why  of  This  Educational  Program,  An  Educational 
Training  Program  in  PPBS,  and  a  Supplement  to  the  Basic 
Educational  Program.  The  supplements  include  pre-  and  post  - 
tests,  a  conceptual  framework,  the  relationship  of  education 
and  economics,  and  methods  of  statistical  analysis  in  PPBS. 
Volume  II  presents  a  simulated  case  problem  based  upon  the 
PPBS  concepts  and  principles  developed  in  Volume  I."   (Author) 

(See  also  citation  in  section  5.0). 

5)  Mushkin,  Selma  J.,  and  James  R.  Cleaveland.   "Planning  for 

Educational  Development  in  a  Planning,  Programming, 
Budgeting  System,"  Interdependence  in  School  Finance: 
The  City-The  State-The  Nation  (k'ashington,  D.C.:  National 
Education  Association,  March-April,  1968),  pp.  59-99. 

This  paper  is  directed  to  school  administrators  in  order  to 
clarify  PPBS  terms  and  concepts.  Also  included  in  this  report 
is  a  schematic  framework  that  has  been  developed  for  the  pur- 
pose of  making  more  informed  policy  choices  for  education. 
This  framework  shows  how  broader  purposes  of  government  can 
be  classified  into  a  hierarchy  of  objectives  and  suitable 
program  structures.   In  addition,  stages  of  analysis,  explana- 
tion of  program  costs,  and  measures  of  effectiveness  are 
presented.  Furthermore,  an  illustrative  program  structure  and 
output  indicators  are  detailed. 

(See  also  section  2.4  for  the  annotation  of  Planning  for  Educa- 
tional Development  in  a  Planning,  Programming,  Budgeting 
System.  Washington,  D.C. :  National  Education  Association, 
1968,  pp.  39-46. 
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6)  Piele,  Philip  K. ,  and  David  G.  Bunting.  Program  Budgeting  for 

the  School  Administrator:  A  Review  of  Dissertations  and 
Annotated  Bibliography.  Eugene,  Oregon:  ERIC  Clearing- 
house on  Educational  Administration,  September,  1969. 

In  this  booklet,  the  authors  have  synthesized  many  significant 
ideas  of  program  budgeting  which  have  arisen  from  doctoral 
dissertations.  Their  purpose  is  to  disseminate  this  informa- 
tion to  school  administrators  who  are  considering  the  possi- 
bility of  adopting  a  program  budget.  Finally,  the  authors 
provide  an  annotated  bibliography  of  the  dissertations  used  in 
their  presentation. 

This  annotation  also  appears  in  section  5.0. 

7)  Sisson,  Roger  L.,  and  Edwin  C.  Brewin.   "An  Introduction  to 

the  Education-Planning-Programming-Budgeting  System." 
Philadelphia:  Government  Studies  Center  of  the  Fels 
Institute,  University  of  Pennsylvania,  November,  1969. 

Sisson  describes  EPPBS  (the  "E"  stands  for  education)  as  a 
method  which  helps  the  school  district  determine  its  present 
situation  and  project  various  ways  in  which  the  school  dis- 
trict could  operate  in  the  future.  An  EPPBS  planning  metho- 
dology is  presented.  An  essential  element  of  EPPBS  is  the 
indicator.   Indicators  are  developed  for  both  inputs  and  out- 
puts under  conditions  of  no  overall  policy  changes  (baseline 
projection)  and  under  policy  adjustment  (the  selection  of  a 
plan).   Indicators  show  gaps  between  present  operations  and 
future  requirements.  These  gaps  become  objectives  from  which 
projects  are  generated  to  close  indicator  gaps.   In  sum,  EPPBS 
permits  the  decision-maker  to  see  the  possible  consequences  of 
"plans  of  operation  in  terms  of  both  indicators  (which  show 
benefits  and  effects  derived)  and  financial  factors  (which 
measure  the  costs)." 

This  annotation  also  appears  in  section  1.4. 

8)  Tenzer,  A.J.,  J.B.  Benton,  and  C.  Teng.   "Applying  the  Concepts 

of  Program  Budgeting  to  the  New  York  City  Police  Depart- 
ment." Santa  Monica,  Calif.:  Rand  Corporation,  June,  1969. 

This  memorandum  describes,  in  a  very  pragmatic  manner,  three 
phases  of  the  PPB  System  developed  for  the  New  York  City 
Police  Department:  the  states  of  developing  a  program-budgeting 
format  for  five  essential  programs,  a  cost-estimating  methodol- 
ogy for  two  major  cost  categories,  and  on-line  computer  model 
for  estimating  cost  implications  of  alternative  resource  allo- 
cations. 

This  annotation  also  appears  in  section  1.5. 
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9)  Wildavsky,  Aaron.   "The  Political  Economy  of  Efficiency,"  The 
Public  Interest  (Summer,  1967),  pp.  30-48. 

Wildavsky  proposes  a  policy  analysis  technique  as  an  alternative 
to  PPBS.  He  analyzes  and  compares  experiences  of  the  Defense 
Department  and  domestic  agencies  with  PPBS.  He  concludes  that^ 
PPBS  has  been  damaging  to  prospects  for  improved  policy  analysis 
at  the  federal  level.  Finally,  he  suggests  ways  to  facilitate 
better  policy  analysis. 

This  annotation  also  appears  in  section  4.1. 

2.3  General  Systems  and/ or  Economic  Analysis 

1 )  A  Study  Design  Prospectus  for  Georgia  Comprehensive  Statewide 

Transportation  Planning.  Prepared  for  the  State  of  Georgia 
State  Planning  Bureau  Contract  #PB182703.  Washington,  D.C.: 
Harold  F.  Wise  &  Associates,  McLean,  Va.:  Alan  M.  Voorhees 
&  Associates,  December,  1968. 

This  technically- oriented  prospectus  is  divided  into  two  chap- 
ters. The  first  chapter  describes  an  approach  to  comprehensive 
statewide  transportation  planning.  The  second  chapter  dis- 
cusses a  proposed  method  for  designing  the  study,  including  a 
suggested  organizational  framework,  as  well  as  discussing 
costs  that  are  involved  in  the  study  design. 

2)  Dror,  Yehezkel.   "Systems  Analysis  for  Development  Decisions: 

Applicability,  Feasibility,  Effectiveness  and  Efficiency." 
Santa  Monica,  Calif.:  Rand  Corporation,  August,  1969. 

Dror  suggests  that  systems  analysis  is  not  adaptable  to  all 
levels  of  problems.   He  proposes  a  meta-analysis  (an  anlysis 
of  the  cases  where  system  analysis  is  most  effective).   He 
presents  a  schema  showing  the  degrees  of  applicability  or 
feasibility  in  using  system  analysis  in  eight  problem  areas, 
one  of  which  is  education.   In  general,  he  concludes  that 
system  analysis  is  more  useful  for  low  or  medium-level  decision- 
making or  in  suboptimizing  cases  of  higher-level  problems. 

This  annotation  also  appears  in  section  3.1. 

3)  Enthoven,  Alain,  C.  The  Simple  Mathematics  of  Maximization, 

Santa  Monica,  Calif.:  Rand  Corporation,  November,  1959. 

This  report  presents  a  thorough,  clear  mathematical  discussion 
of  maximization  for  public  sector  analysis. 
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4)  Forrester,  Jay  W.  "A  Deeper  Knowledge  of  Social  Systems," 

Ekistics,  XXVIII  (April,  1969),  pp.  360-367. 

The  author  suggests  that  complex  social  systems  can  be  viewed 
through  the  use  of  two  variables  (levels  and  rates)  and  the 
resulting  feedback  generated.  Furthermore,  he  states  that 
computer-oriented  models,  based  upon  level  and  rate  schematics, 
should  be  used  to  replace  unreliable,  intuitive  judgements. 
He  describes  the  development  and  use  of  a  complex,  urban 
simulation  model.   In  addition,  Forrester  includes  policy 
changes  into  the  model  and  concludes  that  many  present  day 
policies  may  be  detrimental  to  a  city. 

This  annotation  also  appears  in  section  3.3. 

5)  Hill,  Morris.   "A  Goals-Achievement  Matrix  for  Evaluating 

Alternative  Plans,"  AIP  Journal,  XXXIV  (January,  1968), 
pp.  19-29. 

This  paper  examines  the  efficacy  of  some  established  techniques 
of  plan  evaluation  by  cost/benefit  analysis.  The  presentation 
focuses  on  water  resource  development  projects  and  applications 
of  "balance  sheet  of  development"  to  urban  planning,   it  is 
demonstrated  that  these  techniques  do  not  fully  satisry  tne 
requirements  of  a  rational  planning  process.  An  alternative 
method  of  plan  evaluation,  known  as  the  goals-ach leveraent 
matrix,  is  postulated  and  considered  in  terms  of  requirements 
of  a  rational  planning  process. 

6)  Brandl,  John  E.   "Comment  on  the  Goals-Achievement  Matrix  for 

Evaluating  Alternative  Plans,"  AIP  Journal,  XXXIV  (March, 
1969),  pp.  139-140. 

This  article  is  a  rebuttal  to  Morris  Hill's  article  "A  Goals- 
Achievement  Matrix  for  Evaluating  Alternative  Plans,  Journal 
of  the  American  Institute  of  Planners,  XXXIV,  No.  1  (January, 
1968),  pp.  19-29.   Brandl  suggests  that  Hill's  goals-achieve- 
ment matrix  has  three  deficiencies:  it  fails  to  regard 
diminishing  marginal  utility  and  this  omission  limits  the 
approach  to  linear  objective  functions,  it  does  not  aggregate 
Kits  and  costs  into  a  single  index,  and  the  method  excludes 
the  consideration  of  opportunity  costs. 

While  Brandl  does  not  offer  practical  alternatives,  the  article 
is  useful  as  a  criticism. 

See  also  Hill's  rejoinder  to  Brandl' s  criticism. 
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7)  Hill,   Morris.      "Rejoinder,"  MP  Journal,   XXXIV  (March,   1969), 

pp.   141-142. 

This  is  Hill's  rejoinder  to  John  E.  Brandl's  article,  "Comment 
on  the  Goals-Achievement  Matrix  for  Evaluating  Alternative 
Plans,"  AIP  Journal,  XXXIV  (March,  1969)  pp.  139-140.  Hill 
addresses  himself  to  the  inadequacies  of  the  model  as  posea 
by  Brandl.  He  answers  questions  about  the  criterion  of 
economic  efficiency,  linearity  of  the  model,  opportunity  costs, 
and  the  aggregation  of  costs  and  benefits  into  a  single  index. 

8)  Hitch,  Charles  J.,  and  Roland  N.  McKean  "Efficiency  in  Using 

Defense  Resources,"  The  Economics  of  Defense  in  the  Nuclear 
Age,  (Cambridge,  Mass.:  Harvard  University  Press,  196/;, 
pp.  105-239. 

This  item  refers  to  Part  II  "Efficiency  in  Using  Defense  Re- 
sources" from  the  above-mentioned  book.  The  authors  describe 
methods  to  promote  efficiency  in  military  ^f10"-^1^- 
They  suggest  decentralized  decision-making  and  suboptimizing 
within  f framework  of  incentives  and  criteria.  To  promote 
efficiency,  Hitch  and  McKean  also  suggest  ^^J^et 
mechanism^  into  the  institutional  environment  an  mo   reliance 
upon  quantitative  analysis.  The  authors  inaiccn.e        , 
analysis  is  a  superior  technique  of  evaluation  as  compared  to 
traditional  requirements  and  priorities  yPr°^s'  ,A"  i^ 
strative  example  is  provided  in  Chapter  8.  Hitch  and  McKean 
fully  explain  certain  conceptual  problems  inherent  in  using 
economic  analysis.  They  devote  considerable  space  to  proWems 
of  appropriate  criteria,  incommensurable >  ^ertainty  and 
timeP  Their  experiences  in  public  sector  ^cision  analysis 
would  suggest  considerable  carry-over  to  other  public  sector 
domains. 

9)  Hopkins,  R.C.  "A  Systematic  Procedure  for  System  Development," 
}  IRE  Transactions  on  Engineering_Management  (June,  1961), 

pp.  77-86. 

"A  technique  is  described  for  the  development  of  system  objec- 
tives requirements,  specifications,  and  ^cep^a!  design 
It  is  derived  from  experience  with  a  number  of  actual  systems 
in  the  fields  of  air  defense  and  airbor »%fl^ontrol.  T^e 
need  for  precise  knowledge  of  system  ^"^""J^^^enrs 
stressed   A  checklist  relating  to  environmental  requirements 
is  presented.  A 'seven-step  process  is  described  from  functional 
objectives  to  model  and  test."  (Author) 

This  article  provides  an  excellent  opportunity  to  learn  about 
differences  between  "system  snythesis"  engineering  and  the  more 
usual  writings  about  "system  analysis." 
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10)  Knezevich,  Stephen  J.   "The  Systems  Approach  to  School  Admini- 

stration: Some  Perceptions  on  the  State  of  the  Art  in 
1967,"  Socio-Economic  Planning  Sciences,  II  (1969), 
pp.  127-133. 

"Some  of  the  salient  administrative  activities  consistent  with 
systems  are  clear  delineation  of  objectives;  recognition  of 
the  dynamic  nature  of  goals  and  priorities;  generation  of 
alternative  means  for  allocating  resources;  generation  of  models 
of  the  system  under  study;  utilization  of  quantitatively  oriented 
tools  in  analysis;  and  in  decision  making,  and  employment  of 
interdisciplinary  teams  of  specialists  in  problem  analysis. 
Hard  data  on  the  implementation  of  these  systems  oriented 
approaches  in  education  are  not  available  as  yet. 

School  administrators  in  late  1967  were  on  the  threshold  of 
awareness  of  concepts,  vehicles,  and  procedures  related  to 
systems.  They  have  not  as  yet  perceived  systems  as  an  approach 
to  better  define,  comprehend,  or  resolve  perplexing  educational 
issues  or  operational  problems. 

Those  aspects  of  school  administration  most  amenable  to  opera- 
tions analysis  such  as  transportation,  school  plant  planning 
and  construction,  inventory  control  or  food  services  are  less 
likely  to  be  classified  as  crucial  problems  of  administrators. 
The  more  significant  long-range  policy  problems  in  instruction, 
race  relations,  adapting  the  institution  to  change,  teacher 
militancy,  or  amelioration  of  social  injustices  have  been  con- 
sidered by  too  many  to  be  outside  the  range  of  quantitative 
analysis. 

Not  everyone  talking  about  systems  uses  the  same  data  base. 
The  systems  approach  is  more  likely  to  flourish  where  certain 
conditions  prevail.  Some  pressing  improvements  in  education 
are  needed  to  create  readiness  and  use  of  systems."  (Author) 

11)  Kraft,  Richard  H.P.   Education  and  Economic  Growth.  Tallahassee, 

Fla.:   Florida  State  University,  1968. 

This  reference  is  composed  of  a  series  of  articles  dealing  with 
concepts  relating  education  to  economic  growth.  The  essays 
are  divided  into  two  areas:  Planning  Education  for  Economic 
and  Social  Development,  and  Strategies  of  Human  Resource 
Development.   In  the  former,  topics  such  as  economic  problems 
of  developing  countries  are  delineated;  in  the  latter,  systems 
analysis  in  regard  to  educational  planning  and  organization  is 
among  the  items  covered . 
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12)  Lundin,  G.E.,  and  Gordon  Welty.   "Relevance  of  a  Managerial 

Decision-Model  to  Educational  Administration."  Paper 
presented  at  the  American  Educational  Research  Association, 
Minneapolis,  March,  1970. 

The  authors  describe  some  of  the  inherent  weaknesses  of  the 
rational  or  synoptic  decision  model.  These  weaknesses  limit 
the  model's  usefulness  in  real  management  situations.  They 
propose,  in  a  theoretical  manner,  a  more  realistic  model  which 
remedies  some  of  the  deficiencies  of  the  synoptic  theory  while 
retaining  the  latter' s  analytical  tools. 

13)  McNamara,  James  F.   "A  Mathematical  Programming  Approach  to 

State-Local  Program  Planning  in  Vocational  Education." 
Oregon:   University  of  Oregon,  n.  d. 

McNamara  proposes  the  use  of  a  linear  programming  model  that 
would  enable  state  education  planners  to  evaluate  decisions 
concerning  the  efficient  allocation  of  vocational  education 
funds  to  local  school  districts.   Essentially,  funds  would  be 
allocated  to  maximize  the  number  of  graduates  that  would  fill 
occupational  shortages  in  a  specified  labor  market.  The  author 
explains  the  steps  necessary  to  make  this  manpower-requirements 
approach  effective.   In  the  final  stages  of  the  report,  the 
model  is  applied  to  the  Philadelphia  Labor  Market  (Pennsylvania 
counties  only).  Although  the  model  yields  an  optimum  solution 
within  stated  constraints,  it  does  not  make  the  decision. 
Rather,  it  allows  the  decision-maker  to  be  much  clearer  about 
the  implications  of  different  alternatives. 

14)  Sisson,  Roger  L.   "Applying  Operational  Analysis  to  Urban  Educa- 

tional Systems."  Philadelphia:  Management  Science  Center, 
University  of  Pennsylvania,  January,  1967. 

Sisson  explains  the  nature  of  operational  analysis  and  how  it 
can  be  applied  to  a  complex  system  such  as  an  urban  school 
district.  The  author  lists  and  discusses  five  stages  of 
operational  analysis:   system  identification,  description  of 
the  subsystems,  processes,  flows,  and  decisions,  the  develop- 
ment of  a  computer  simulation  model  (subordinate  models, 
measuring  performance  attributes  and  behavior  phenomena,  are 
also  needed),  validation  of  the  model  by  school  tests,  and 
uses  of  the  model  to  assess  system  characteristics  and  policy 
changes . 

Note:   Sisson  has  applied  some  of  his  thinking  to  problems  in 
the  Philadelphia  School  District. 
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15)  Smithies,  A.  Government  Dec is ion- Making  and  the  Theory  of 
Choice.   Santa  Monica,  Calif.:  Rand  Corporation,  1964. 

The  author  presents  a  cost-effectiveness  theory  based  on 
indifference  curves.  His  article  highlights  many  theoretical 
problems  which  are  often  not  faced  by  analysts  when  they  attack 
real  problems. 

2 .4  Cost-Effectiveness  Analysis 

1)  Chuang,  Ying  C. ,  and  John  B.  Peper.   "The  Philadelphia  Model 

for  Title  I  Evaluation."  Philadelphia:   School  District 
of  Philadelphia,  April,  1969. 

The  authors  describe  the  development  of  an  evaluation  model 
for  Title  I  projects  in  the  Philadelphia  School  System.  The 
evaluation  model  has  two  major  divisions:  an  individual 
project  evaluation  model,  and  an  overall  cost-effectiveness _ 
model.  The  latter  was  formulated  so  that  meaningful  compari- 
sons could  be  made  among  projects.  The  authors  present  a 
theoretical  discussion  of  each  section  of  the  evaluation  model. 

This  annotation  also  appears  in  section  2.1. 

2)  Fox,  Peter  D.   "A  Theory  of  Cost-Effectiveness  for  Military  Sys- 

tems Analysis,  "  Journal  of  the  Operations  Research  Society 
of  America,  XIII  (March-April,  1965),  pp.  191-201. 

"This  paper  presents  a  theoretical  basis  for  cost-effectiveness 
analysis.   It  is  argued  that,  frequently,  a  range  of  effective- 
ness or  cost  levels  may  be  acceptable  to  whoever  must  ultimately 
decide  which  military  system  (if  any)  should  be  acquired.  The 
function  of  the  analyst  is  to  present  a  schedule  of  alterna- 
tives and  not  to  optimize  in  the  sense  that  he  recommends  the 
selection  of  a  particular  alternative.  The  formulation  of  the 
schedule  is  discussed  where  the  cost  and  effectiveness  associated 
with  each  alternative  are  viewed  as  random  variables.  The 
paper  concludes  with  some  general  observations  relating  to 
military  system  selection."  (Author) 

Fox's  article  treats  some  very  general  theoretical  aspects  of 
cost-effectiveness  but  does  little  to  add  to  such  prescriptions 
as  "select  the  action  which  is  expected  to  yield  the  highest 
return  for  a  given  level  of  cost." 
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3)  Heuston,  M.C.,  and  G,  Ogawa .   "Observations  on  the  Theoretical 

Basis  of  Cost-Effectiveness,"  Journal  of  the  Operations 
Research  Society  of  America,  XIV  (March-April,  1966), 
pp.  242-266. 

"This  paper  presents  some  observations  on  the  theoretical 
foundation  of  cost-effectiveness  analysis.   It  describes  the 
results  of  continuing  research  to  develop  a  comprehensive  and 
rigorous  description  of  the  important  elements  of  cost-effective- 
ness as  used  by  the  aerospace  industry  for  military  and  commer- 
cial systems  planning.   The  primary  objective  is  to  utilize 
basic  mathematical  and  statistical  theory  to  construct  the 
rules,  properties,  and  hypotheses  that  are  needed  to  satisfy 
the  contractual  requirements  imposed  by  various  government 
customers."  (Author) 

4)  Peper,  John  B.   "Summary  of  Program  Relevance  Evaluation  Model." 

An  Address  prepared  for  the  American  Educational  Research 
Association,  Minneapolis,  March,  1970. 

This  paper  may  be  viewed  as  a  companion  to  the  paper  by  Ying 
Chuang  and  Peper.   The  major  purpose  of  Peper' s  paper  is  to 
report  the  development  and  use  of  the  model  in  Philadelphia 
rather  than  a  detailed  description  of  the  model  itself.  The 
discussion  treats  the  assignment  of  priorities,  in  the  form 
of  relative  weights,  to  projects. 

Peper  concludes  that  the  model  appears  to  be  useful  in  program 
planning  and  in  making  new  funding  decisions. 

This  annotation  also  appears  in  section  1.3. 

5)  Planning  for  Educational  Development  in  a  Planning,  Programming, 

Budgeting  System.  Washington,  D.C.:   National  Education 
association,  1968. 

This  article  (pages  39-46)  is  included  to  indicate  how  cost- 
effectiveness  analysis  was  applied  to  child  care  programs.  The 
author's  planning  process  included  generating  a  series  of 
optional  programs,  formulating  measures  of  progress  toward 
satisfying  the  objective,  quantifying  the  programs,  in  terms 
of  effectiveness  measures  and  assessing  the  additional  cost 
implied  by  optional  programs.  The  author  clearly  demonstrates 
the  types  of  information  that  have  resulted  from  this  analysis. 
The  author  suggests  that  the  same  type  of  analytical  approach 
could  be  used  for  similar  full-scale  approaches. 

(See  also  section  2.2  for  the  annotation  of  Mushkin  and  Cleave- 
land's  article  "Planning  for  Educational  Development  in  a 
Planning,  Programming,  Budgeting  System,"  found  in  Interdepend- 
ence  in  School  Finance:  The  City  -  The  State  -  The  Nation.) 
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6)  Temkin,  Sanford.   "A  Comprehensive  Theory  of  Cost-Effective- 

ness." Philadelphia:  Research  for  Better  Schools,  Inc., 
April,  1970. 

The  author  notes  two  deficiencies  that  stem  from  contemporary 
cost-effectiveness  methods.  Each  technique  suffers  from  a 
lack  of  a  firm  theoretical  base  of  assumptions,  and  an  absence 
of  a  means  for  selecting  an  efficiency  technique  for  a  practi- 
cal problem.  This  paper  is  intended  to  correct  these  inade- 
quacies and  to  appeal  to  the  mathematically- trained  person  who 
is  interested  in  a  general  theory  of  school  system  planning 
and  in  methods  such  as  cost/benefit  and  cost-effectiveness 
analysis.  Temkin  presents  a  theoretical  development  of  9 
decision  cases  in  an  evolutionary  framework  ranging  from  the 
simplest  decision-making  situation  (Case  1)  to  complex  circum- 
stances faced  by  school  administrators  (Cases  8  and  9).  The 
cases  are  augmented  by  mathematical  proofs.   In  addition,  the 
author  discusses  his  rationale  for  recommending  the  use  of 
cost-effectiveness  for  school  district  comprehensive  planning 
rather  than  benefit/cost  analysis. 

This  annotation  also  appears  in  section  2.1. 

7)  Technomics,  The  Feasibility  of  Cost-Effectiveness  Analysis  for 

Title  I  Public  Law  89-10,  Santa  Monica,  Calif.:  Technomics, 
Inc.,  1966. 

This  report  describes  one  of  the  earliest  practical  applica- 
tions of  cost-effectiveness  technology  to  school  systems.  The 
school  systems  of  12  cities  were  studied  with  regard  to  their 
expenditure  of  Title  I  funds.  Within  this  volume  are  provided 
a  general  background  of  cost-effectiveness,  the  development  and 
reasoning  underlying  the  methodology  used,  some  survey  findings 
from  the  impact  of  Title  I,  some  needed  modifications  in  cost- 
effectiveness  methods  to  make  them  more  applicable  to  school 
systems,  and  a  workable  PPBS  tool  for  education.  The  authors 
conclude  that  cost-effectiveness  can  be  applied  to  educational 
expenditures,  provided  some  modifications  are  made  in  its 
technology. 

8)  Zabronsky,  Herman.   "A  Mathematical  Theory  of  Cost-Effective- 
ness," Socio-Economics  Planning  Sciences,  I  (September, 
1967),  pp.  3-18. 

"Planning-programming-budgeting  systems  are  applied  where  more 
precise  comparison  of  alternate  plans  and  programs  is  desired 
than  can  be  achieved  by  traditional  methods.   Cost-effective- 
ness analysis  is  used  to  make  such  comparisons  by  relating  re- 
source input  and  useful  output  of  a  program  relative  to  stated 
planning  goals.  Proper  application  of  the  techniques  of  cost- 
effectiveness  depends  on  their  mathematical  foundation.   In 
this  paper  a  rigorous  mathematical  theory  of  cost-effectiveness, 
based  on  the  Gradient  Method,  is  formulated  and  developed.   Com- 
putation of  cost-effectiveness  measures  is  considered  and 
procedures  for  determining  optima  are  derived."  (Author) 
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2.5  Input-Output  Analysis 


1)  Dyer,  Henry  S.   "The  Conceptand  Utility  of  Educational  Per- 

formance Indicators."  Paper  presented  at  the  Systems 
Science  and  Cybernetics  Conference,  Boston,  October,  1967. 

The  author  describes  a  technique  for  the  development  of  edu- 
cational  performance  indicators  based  upon  regression  analysis. 
A  theoretical  model  is  presented  to  the  reader  with  suggestions 
and  interpretations  of  its  use.  Possible  problems  of  implemen- 
tation are  discussed.   Finally,  Dyer  implies  that  educational 
performance  indicators  may  "provide  a  basis  for  genuine  cost- 
benefit  analysis." 

This  annotation  also  appears  in  section  1.4. 

2)  Kershaw,  J. A.,  and  R.N.  McKean.   "Systems  Analysis  and  Educa- 

tion."  Santa  Monica,  Calif.:  Rand  Corporation,  1959. 

This  was  one  of  the  first  articles  suggesting  input-output 
analysis  for  educational  decision-making. 

3)  Levin,  Henry  M.   "A  New  Model  of  School  Effectiveness."  Stan- 

ford, Calif.:   Stanford  University,  May,  1970. 

"In  recent  years  there  has  emerged  a  number  of  large-scale 
statistical  studies  exploring  the  effects  of  schools  on  achieve- 
ment.  In  virtually  all  cases  these  studies  explored  the 
relationships  between  a  single  output  on  the  one  hand,  and 
student  and  school  characteristics  on  the  other.   Such  an 
approach  can  not  take  into  account  the  fact  that  schools  pro- 
duce several  outputs  simultaneously. 

In  this  effort  schools  are  viewed  as  multiproduct  firms  that 
add  to  the  "capital  embodiment"  of  children,  in  turn  affecting 
their  skills  and  attitudes.  A  four-equation  system  is  posited 
for  determining  simultaneously  the  effects  of  school  and  other 
variables  on  sixth  graders'  verbal  achievement,  sense  of 
efficacy,  grade  level  aspiration,  and  parents'  educational 
aspirations  for  their  children.  The  model  is  estimated  for 
some  600  sixth  graders  in  a  large  eastern  city.  One  of  the 
important  findings  is  the  suggestion  that  earlier  studies  may 
have  over-stated  substantially  the  direct  effect  of  background 
characteristics  on  scholastic  achievement."  (Author) 
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4)  Miller,  Richard  I.   "A  System  Analysis  of  Education  in  Kentucky 

Public  Schools."  Paper  presented  at  the  American  Educa- 
tional Research  Association's  Annual  Meeting,  Chicago, 
February,  1968. 

The  purpose  of  this  article  is  to  analyze  the  relationships 
between  school  variables,  both  individually  and  in  combinations. 
Multiple  regression  analysis  was  employed  to  examine  the  re- 
lationships among  socio-economic  (input)  factors,  instructional 
(process)  factors,  and  several  measures  of  student  achievement 
(output).   Nineteen  factors  were  considered.  Miller  lists  the 
12  findings  of  the  study  and  provides  an  explanation  of  results. 
Although  the  author  states  that  the  Kentucky  study  may  not  be 
applicable  to  other  states,  some  of  the  findings  do  not  sub- 
stantiate preconceived  notions  of  relationships  among  variables. 

5)  Mood,  Alex  M.  "Macro-Analysis  of  the  American  Educational 

Systems,"  Operations  Research,  XVII  (September-October, 
1969),  pp.  770-784. 

"This  paper  presents  a  rudimentary  model  of  the  public  school 
system  as  an  input-output  process.  The  inputs  are  students' 
own  abilities  and  attitudes,  parental  support,  peer  support, 
quality  of  the  school  system,  community  support,  and  society's 
posture  with  respect  to  education.   Outputs  are  various  cate- 
gories of  academic  achievement  as  well  as  social  competence, 
responsibility,  self  confidence,  creativeness,  ethics,  and 
ambition.  All  these  factors  must  be  measured  by  index  numbers 
or  simple  indicators.   The  model  is  a  set  of  regression  equations 
relating  outputs  to  inputs.   Some  implementation  of  the  model 
has  been  made  possible  by  means  of  data  gathered  in  the  U.S. 
Office  of  Educations 's  Equality-of  Educational-Opportunity 
Survey."   (Author) 

This  annotation  also  appears  in  section  1.4. 

6)  Mort,  Paul  R. ,  and  Orlando  F.  Furno.   Theory  and  Synthesis  of  a 

Sequential  Simplex:  A  Model  for  Assessing  the  Effectiveness 
of  Administrative  Policies.   New  York:  Teachers  College, 
Columbia  University,  1960, 

This  monograph  represents  one  of  the  first  "input-output" 
methods  in  the  literature  of  educational  administration.   The 
monograph  presents  the  theory  of  Sequential  Simplex,  its 
associated  models  and  practical  applications  for  school  admini- 
strators.  Essentially,  the  model  attempts  to  control  gross 
differences  that  exist  between  school  systems  (wealth,  size, 
population,  board  policy  and  personnel)  in  order  to  focus  its 
attention  on  overlooked  subtle  differences  (administrative 
policy,  legal  structure  and  organization).  The  authors  describe 
how  a  given  factor  (input),  in  its  interplay  with  others, 
affects  the  quality  (output)  of  the  system.   Furno  and  Mort 
list  and  describe  11  steps  for  the  measurement  of  the  impact  of 
a  factor.  Definitions  and  statistical  methods  are  contained 
in  supplements  to  the  monograph. 
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2.6  Cost/Benefit  Analysis 


1)  Hamburg,  Morris.   "Statistical  Decision  Theory  and  Benefit- 

Cost  Analysis  for  Preferredness  of  Choice  Among  Alterna- 
tive Projects,"  Psychology  of  Management  Decision,  1967, 
pp.  44-60. 

"The  purpose  of  this  paper  is  to  suggest  a  £generalj  model 
which  combines  benefit/cost  analysis  with  statistical  decision 
theory,  particularly  for  public  expenditure  decisions."  By 
adapting  Bayesian  statistics  to  the  traditional  model  of 
benefit/cost  analysis,  the  resulting  method  is  able  to  handle 
problems  such  as  postponability  of  investment,  uncertainty  and 
interdependence  of  projects.   In  the  final  sections  of  the 
article,  Hamburg  proposes  that  the  opportunity  cost  of  a 
project  be  used  as  the  discount  rate. 

2)  McKean,  Roland  N.   "Special  Problems  in  the  Analysis  of  Water- 

Resource  Projects,"  Efficiency  in  Government  Through 
Systems  Analysis,  David  B.  Hertz,  editor  (New  York: 
John  Wiley  &  Sons  Inc.,  1965),  pp.  103-182. 

This  item  refers  to  Part  III,  "Special  Problems  in  the  Analysis 
of  Water-Resource  Projects."  McKean  discusses  criteria  in 
the  selection  of  water-resource  projects,  kinds  of  benefits 
and  costs,  such  as  spillovers,  over-counting  and  secondary 
benefits,  and  valuation  of  benefits.  The  author  describes  the 
deficiencies  of  the  traditional  cost/benefit  ratio  measure  as 
a  decision  criterion.   McKean  recommends  the  use  of  the  inter- 
nal rate  of  return  criterion.   He  discusses  the  nature  of  an 
appropriate  discount  rate  (not  the  market  rate)  and  describes 
how  time  preferences  of  decision-makers  can  alter  optimum 
decisions. 

In  viewing  types  of  benefits  and  costs,  McKean  recommends  that 
technological  spillovers,  but  not  pecuniary  effects,  be  in- 
cluded in  project  analysis.   In  addition,  secondary  benefits 
only  arise  under  situations  of  previous  unemployment.   McKean 
also  mentions  the  treatment  of  forgone  taxes  and  other  mis- 
cellaneous double  counting  that  might  mislead  the  analyst. 

In  the  final  section  of  Part  III,  the  author  presents  a 
theoretical  model  as  well  as  practical  applications  of  his 
methodology  for  valuing  benefits.   Examples  include  cases 
involving  incremental  and  indivisible  amounts  of  investment. 

3)  Polin,  Lewis.   "Benefit/Cost  Analysis."  Philadelphia: 
Research  for  Better  Schools,  1968. 

This  paper  assesses  benefit/cost  analysis  from  several  vantage 
points:   its  history,  the  problems  involved  in  its  application, 
some  illustrative  examples  of  its  use,  and  its  present  role  in 
dec  is  ion-making . 
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4)  Proposed  Practices  for  Economic  Analysis  of  River  Basin  Projects, 
Report  to  the  Federal  Inter-Agency  River  Basin  Committee 
Prepared  by  the  Sub- Committee  on  Benefits  and  Costs 
(Washington:  1950). 

This  booklet  is  often  referred  to  as  the  "green  book."  It  is 
one  of  the  first  practical  treatments  of  "what  to  do"  in 
analyzing  benefits  and  costs  for  public  sector  projects. 

In  discussing  criteria  at  the  project  level,  the  green  book 
suggests  the  criterion  of  maximization  of  net  benefits  for 
comparing  different  projects. 
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3.0  SYSTEM  CHARACTERISTICS  AND  QUALITIES 

3.1  Feasibility 

1)  Caldwell,  Michael  S.   "An  Approach  to  the  Assessment  of  Educa- 

tional Planning,"  Educational  Technology,  (October,  1968), 
pp.  5-12. 

This  article  provides  a  framework  assessing  the  overall  feasi- 
bility of  alternative  plans.  A  full  explanation  of  illustrative 
evaluative  criteria  is  presented. 

2)  Dror,  Yehezkel.   "Systems  Analysis  for  Development  Decisions: 

Applicability,  Feasibility,  Effectiveness  and  Efficiency." 
Santa  Monica,  Calif. :  Rand  Corporation,  August,  1969. 

Dror  suggests  that  systems  analysis  is  not  adaptable  to  all 
levels  of  problems.   He  proposes  a  meta-analysis  (an  analysis 
of  the  cases  where  system  analysis  is  most  effective).   He 
presents  a  schema  showing  the  degrees  of  applicability  of 
feasibility  in  using  system  analysis  in  eight  problem  areas, 
one  of  which  is  education.   In  general,  he  concludes  that 
system  analysis  is  more  useful  for  low  or  medium-level  decision- 
making or  in  suboptimizing  cases  of  higher-level  problems. 

This  annotation  also  appears  in  section  2.3. 

3)  Friedly,  Philip  H.,  et  al.   "Benefit-Cost  Applications  in  Urban 

Renewal:  A  Feasibility  Study."  Clearinghouse  for  Federal 
Scientific  and  Technical  Information  Contract  #182969, 
Bethesda,  Md.:  Resource  Management  Corporation,  August, 
1968. 

This  item  refers  to  Chapter  6,  "Evaluating  the  Impacts  of 
Renewal  Activities,"  of  this  report.  A  matrix  is  developed 
showing  the  interactions  between  11  urban  renewal  objectives 
and  11  welfare  indicators.  Measuring  these  11  welfare  in- 
dicators would  yield  information  regarding  the  attainment  of 
the  specified  objectives.   Nevertheless,  in  their  discussion 
of  each  of  the  welfare  indicators,  the  authors  point  out  that 
the  criteria  are  not  necessarily  commensurable.   Empirical 
testing  must  now  provide  information  about  usefulness  and 
administrative  costs  for  the  respective  indicators. 

This  annotation  also  appears  in  section  1.4. 


-53-    CPL  Exchange  Bibliography  #219 

4)  Hodgkinson,  Christopher.   "Organizational  Influence  on  Value 
Systems."  Paper  presented  at  the  American  Educational 
Research  Association  Annual  Meeting,  Minneapolis,  March, 
1970. 

"This  paper  examines  the  relationship  between  organizational 
climate  as  operationally  defined  by  A.W.  Halpin  and  values  held 
by  teaching  staffs  and  administrators.  Values  are  measured 
empirically  on  the  instrument  developed  by  W.A.  Scott,  author 
of  Values  and  Organizations.  The  purpose  is  to  explore  the  ways 
in  which  values,  through  the  mediating  factor  of  perception, 
are  related  to  organizational  climate  and  to  test  a  number  of 
hypotheses  and  questions  bearing  on  this  relationship:  e.g. 
Do  values  change  as  a  result  of  promotion  in  the  hierarchy? 

The  position  is  developed  that  organizations  do  in  fact  influ- 
ence individual  value  systems  although  the  influence  is  bifid 
and  interactional.  The  empirical  research  suggests  a  theory  of 
"organizational  value"  which  can  be  outlined  in  brief. 

The  validity  of  the  OCDQ*  instrument  is  challenged.  This 
instrument  is  still  being  employed  in  a  variety  of  research 
projects  and  its  use  may  be  inadvisable.  The  exploration  of 
the  values/organization  relationship  is  especially  pertinent 
to  education  considered  as  a  value  propagating  institution." 
(Author) 

*Note:  OCDQ  refers  to  the  Halpin  Organizational  Climate  Descrip- 
tion Questionnaire.  "Such  an  instrument  yields  a  measure  of 
eight  dimensions  of  social  perception  on  the  part  of  members  of 
an  education  organization.  Four  of  these  measures  have  to  do 
with  inter- staff  perceptions  (Intimacy,  Disengagement,  Esprit, 
Hindrance)  and  four  have  to  do  with  staff  perceptions  of  the 
administration  of  the  organization  (Thrust,  Consideration, 
Aloofness,  Production  Emphasis)."  For  a  fuller  discussion  of 
this  scale,  see  Halpin,  A.W.  Theory  and  Research  in  Admini- 
stration (1966). 

3.2  Management  Information  Systems 

1)  A  Study  Design  for  a  Comprehensive  Planning  Program  in  the 

Denver  Region.  Clearinghouse  for  Federal  Scientific  and 
Technical  Information  Contract  #6808.   Denver:  Denver 
Regional  Council  of  Governments,  December  1968. 

This  item  specifically  pertains  to  the  fourth  chapter  of  the 
Denver  Regional  Council  of  Governments  comprehensive  plan.  This 
chapter  discusses  the  development  of  a  system  for  managing 
those  studies  that  are  to  be  used  as  inputs  to  the  overall 
planning  process.   Included  are  descriptions  of  manpower  and 
cost  requirements,  programming  and  scheduling  of  activities, 
implications  with  respect  to  organization,  and  the  development 
of  a  management  information  system. 

This  annotation  also  appears  in  section  1.5. 
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2)  Bratten,  J.E.,  L.T.  Krebs,  and  F.A.  Yett.   "The  Development  of 

a  School  District  Budget  Planning  System."  Santa  Monica, 
Calif.:   System  Development  Corporation,  May,  1968. 

"The  design  and  initial  trial  phases  of  a  project  to  develop 
SPLAN  (pronounced  "es-plan")  --  a  computer-based  planning 
system  for  school  districts  --  are  reported  in  this  document. 
The  work  described  was  accomplished  during  the  period  1  March 
1967  through  29  February  1968. 

An  initial  version  of  SPLAN  was  made  available  to  administrators 
at  Charter  Oak  Unified  School  District  from  May  1967  until  the 
present.   SPLAN  offers  the  user  two  capabilities:   from  a 
teletype  at  the  district  office  linked  to  a  computer  at  System 
Development  Corporation,  the  user  can  retrieve  specific  portions 
or  summaries  of  a  prestored  description  of  his  personnel;  in 
addition,  the  user  can  prepare  alternative  budgets  at  the 
teletype  by  modifying  factors  such  as  the  salary  base,  salary 
schedule,  etc. 

Analyses  of  data  and  observational  notes  indicate  the  need  for 
two  revisions  to  SPLAN.  One  is  to  incorporate  SPLAN  more 
directly  into  the  budget  planning  cycle  by  regularly  providing 
information  to  administrators.   The  other  is  to  delete  the 
interactive  (teletype)  capability;  administrators  making 
planning  decisions  can  readily  tolerate  several  days'  delay  in 
obtaining  the  results  of  those  decisions."  (Author) 

3)  Brooks,  Elizabeth  A.  (ed.)  MSEIP  Documentation  of  Project 

Development  and  General  System  Design.  Des  Moines,  Iowa: 
Midwestern  States  Educational  Information  Project,  1969. 

The  Midwestern  States  Educational  Information  Project  (MSEIP) 
was  a  combined  effort  of  13  state  educational  agencies  (SEAs) 
to  develop  and  implement  an  integrated  information  system, 
which  reflected  the  need  for  timely  and  accurate  information 
for  decision-making.  The  organizational  structure  of  the 
Project  was  significant  in  that  an  active  working  relationship 
was  established  between  the  Federal  Government,  SEAs  and  local 
educational  agencies.   (Taken  from  Forward  of  report) 

4)  Cummings,  Peter  C.   "A  Review  and  Analysis  of  Decision  Making 

with  a  View  Toward  a  Comprehensive  Management  Information 
System."  Philadelphia:  Research  for  Better  Schools,  May, 
L970. 

This  paper  is  addressed  to  decision-making  and  decision-analysis 
in  connection  with  Integrated  Municipal  Information  Systems 
(IMIS).   Sections  are  devoted  to  categories  of  decisions, 
decision  process  models,  decision- tree  analysis,  and  decision 
analysis  methodology. 
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5)  Guertin,  Wilson  H.   "Straight  Talk  About  Computer  Information 

Systems,"  Educational  Technology,  IX  (1969),  p.  25. 

This  article  describes  the  nature  of  computer  information 
systems  and  some  of  the  difficulties  arising  from  their  imple- 
mentation in  educational  environments.   Guertin  defines  four 
types  of  information  systems:  a  management  information  system 
(for  sound  decision-making),  a  pupil  information  system  (to 
improve  teacher-pupil  situations),  a  guidance  information  system 
(for  pupil  understanding),  and  a  total  information  system. 
Furthermore,  the  author  states  that  educators  must  make  a 
commitment  to  the  information  system's  overall  design  both  as 
to  "software"  and  "hardware."  Guertin  recommends  that  existing 
systems  improve  standardization  and  uniformity,  that  the  Federal 
government  fund  software  development,  and  that  schools  and 
universities  combine  to  implement  total  information  systems. 

6)  Sandberg,  John  (ed.)  Schools  and  Comprehensive  Urban  Planning. 

Portland,  Oreg. :  Northwest  Regional  Educational  Laboratory, 
September,  1969. 

The  purpose  of  this  document  is  to  provide  ideas  that  will  help 
school  administrators  work  more  effectively  with  local,  state 
and  regional  planning  bodies.  The  authors  stress  the  urgency 
for  comprehensive,  interagency  planning  in  urban  areas.  A 
series  of  articles  examine  some  aspects  of  integrative,  coopera- 
tive planning.   For  example,  John  Sandberg' s  article  assesses 
some  of  the  problems  that  beset  agencies  that  try  integrative 
planning.   In  an  article  entitled,  "Urban  Planning  Objectives," 
the  authors  list  certain  "do's  and  don'ts"  for  involving 
citizens  in  the  planning  process.   They  also  state  that  the 
quality  of  public  participation  depends  largely  on  the  people's 
access  to  facts  and  ideas.  The  latter  two  articles  by  Hansen 
and  Musoff  discuss  planning  models  (types  of  and  strategies 
used  to  bring  about  change)  and  the  stages  in  the  development 
of  an  information  system  for  urban  areas. 

This  annotation  also  appears  in  sections  2.1  and  4.1. 

7)  Yarbrough,  Everett  L.   "The  Florida  Plan  for  a  Statewide  Educa- 

tion Management  Information  System,"  AEDS  Journal,  III 
(September,  1969),  pp.  1-9. 

The  author  states  that  there  is  a  need  for  a  management  informa- 
tion system  for  Florida.  The  Florida  Educational  Management 
Information  System  (FEMIS)  has  been  recommended  to  the  state 
legislature  for  implementation.  The  system  would  provide  an 
uninterrupted  flow  of  information  among  all  educational  agencies 
within  the  state. 


-56-    CPL  Exchange  Bibliography  #219 

3.3  Feedback 

1)  Automatic  Control.   New  York:   Simon  and  Schuster,  1955. 

This  series  of  12  essays  represents  one  of  the  earliest  attempts 
to  provide  "a  comprehensive  picture  of  the  unifying,  essential 
principles  of  automatic  control."  For  example,  in  his  essay 
"feedback,"  Tustin  views  feedback  as  the  common  pattern  that 
underlies  the  various  phenomena  of  life.  Ayes,  Pease,  and 
Lessing  discuss  the  automatic  chemical  plant,  machine  tool, 
and  office,  respectively.   In  the  final  sections  of  the  book, 
King  considers  how  information  can  be  expressed  in  mathematical 
terms,  while  in  an  article  entitled  "The  Mathematics  of  Infor- 
mation," Weaver  shows  how  the  property  of  entropy  and  laws  of 
thermodynamics  can  be  applied  to  communication  concepts. 

2)  Forrester,  Jay  W.   "A  Deeper  Knowledge  of  Social  Systems," 
Ekistics,  XXVIII  (April,  1969),  pp.  360-367. 

The  author  suggests  that  complex  social  systems  can  be  viewed 
through  the  use  of  two  variables  (levels  and  rates)  and  the 
resulting  feedback  generated.   Furthermore,  he  states  that 
computer-oriented  models,  based  upon  level  and  rate  schematics, 
should  be  used  to  replace  unreliable,  intuitive  judgments. 
Some  effort  is  expended  in  describing  the  development  and  use 
of  a  complex,  urban  simulation  model.   In  addition,  Forrester 
includes  policy  changes  in  the  model  and  concludes  that  many 
present  day  policies  may  be  detrimental  to  a  city. 

This  annotation  also  appears  in  section  2.3. 

3 .4  Organization  Structure 

1)  Andes,  John,  Roe  L.  Johns,  and  Ralph  B.  Kimbrough.  Alternative 
Models  for  Organization  and  Administration  of  Urban  School 
Systems,  Working  Paper  #4,  U.S.  Office  of  Education  Re- 
search  Project  #8-0254.  Gainesville,  Fla.:   University  of 
Florida,  February,  1970. 

This  article  focuses  on  two  basic  areas  of  U.S.O.E.  Research 
Project  #8-0254  (University  of  Florida).  This  report  describes 
in  depth  some  existing  models  of  urban  school  organization, 
and  it  also  provides  an  extensive  discussion  of  alternative 
organization  and  administrative  models  for  urban  school  systems. 
Three  bureaucratic  urban  school  organizational  models  (con- 
solidation, centralization  and  decentralization)  are  treated 
and  their  sub-types.  Alternative  models  discussed  include  the 
pluralistic  models  (Federal,  Egalitarian,  PPBS).   Comments 
indicating  some  of  the  justification  and  necessary  conditions 
for  their  implementation  are  also  provided.   Finally,  the  report 
deals  briefly  with  the  establishment  of  better  community-school 
communication.  A  mechanism  called  the  System  Spanning  Unit 
serves  to  provide  this  communication  interface. 

This  annotation  also  appears  in  section  4.1. 
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2)  Abbott,  Max  G.   "The  School  as  a  Social  System:   Indicators 

for  Change,"  Socio-Economic  Planning  Sciences,  II  (1969), 
pp.  167-174. 

"In  systems  analysis,  the  school  may  be  viewed  as  a  functionally 
differentiated  subsystem  of  the  broader  social  system  in  which 
it  is  embedded.  To  maintain  a  viable  relationship  with  that 
social  system,  the  school  is  subject  to  continual  changes  to 
meet  the  shifting  social,  economic,  political,  and  technological 
forces  in  its  environment.   However,  the  more  successful  the 
school  organization  is  in  accurately  assessing  changing  environ- 
mental forces,  and  in  making  appropriate  adjustments  to  those 
forces,  the  more  successful  will  it  be  in  resisting  temporary 
pressures  and  transitory  movements  and  in  controlling  its  own 
directions. 

A  major  research  problem  is  to  identify  those  organizational 
properties  that  enable  the  school  to  assess  accurately  new 
demands  and  to  adjust  appropriately  to  those  demands.  Theoreti- 
cally, those  properties  might  be  expected  to  include:  1)  oper- 
ationalized  statements  of  instrumental  goals;  2)  a  work 
structure  that  involves  interdependence  in  task  performance; 
3)  participation  in  decision  making;  4)  an  incentive  system 
that  utilizes  performance  criteria  rather  than  expressive 
relationships;  5)  personnel  practices  that  encourage  a  cos- 
mopolitan orientation;  and  6)  institutionalized  provisions  for 
change  advocacy. 

The  specification  of  the  relationships  among  these  properties, 
or  variables,  and  the  determination  of  means  for  assessing 
them  quantitatively,  are  tasks  that  remain  to  be  accomplished." 
(Author) 

3)  Baker,  Walter.   "Management  by  Objectives:  A  Philosophy  and 

Style  of  Management  for  the  Public  Sector,"  Canadian  Public 
Administration,  XII  (Fall,  1969),  pp.  427-443. 

This  article  assesses  the  suitability  of  "management  by  objec- 
tives" for  both  private  and  public  sectors.   Baker  lists  a 
set  of  six  techniques  that  can  be  used  for  the  implementation 
of  "managing  by  results."  Although  the  author  presents  some 
of  the  dangers  or  pitfalls  that  may  arise  out  of  management  by 
objectives,  he  concludes  that  its  adoption  would  lead  to  four 
benefits:  more  systematic  planning,  focusing  on  those  objec- 
tives that  involve  problem-solving  and  creativity,  achieving 
necessary  decentralization,  and  reviewing  executive  performance. 
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4)  Elam,  Stanley,  and  Gordon  I.  Swanson  (eds.)  Educational  Plan- 

ning in  the  United  States.   Itasca,  111.:   F.E.  Peacock, 
Inc.,  1969. 

A  series  of  articles  from  the  Second  Symposium  on  Educational 
Requirements  for  the  1970' s,  An  Interdisciplinary  Approach 
held  at  the  University  of  Minnesota  is  presented.  The  essays 
deal  with  various  aspects  of  educational  planning.  Discussions 
of  each  of  the  papers  are  included. 

For  example,  there  is  an  article  by  Philip  G.  Smith  entitled 
"Objectives  for  American  Education,"  a  paper  entitled  "The 
Status  of  Educational  Planning  in  the  United  States,"  written 
by  Francis  S.  Chase  and  "The  Planning  Process,"  by  Kjell  Eide. 
In  the  latter  regard,  the  author  explains  the  functions  of  an 
organization,  such  as  programming  and  implementation,  and 
suggests  that  organizational  structure  should  be  flexible  enough 
to  account  for  the  interplay  among  administrative  functions. 
Furthermore,  Eide  argues  for  a  separate  planning  unit  which 
would  be  able  to  identify  discrepancies  that  exist  between 
the  objectives  and  policy  instruments  of  separate  branches  of 
the  organization  and  those  objectives  and  policy  instruments 
that  encompass  the  entire  institution.   In  the  final  stages  of 
the  article,  Eide  details  a  method  for  minimizing  role  con- 
flicts and  communication  distortion  that  may  occur  between  the 
executive  and  planning  branches  of  the  organization. 

5)  Jensen,  Gale  Edward.  Problems  and  Principles  of  Human  Organi- 

zation in  Educational  Systems.  Ann  Arbor,  Mich. :  Ann 
Arbor  Publishers,  May,  1969. 

The  major  premise  of  this  book  is  that  an  effective  instructional 
environment  and  a  productive  system  characterized  by  high  morale 
become  possible  when  organizational  problems  are  handled  well 
by  school  system  members.   This  book  is  intended  to  suggest 
to  the  school  administrator  skills  that  are  necessary  in  dealing 
with  organizational  problems.  Organizational  concepts  are  pre- 
sented and  case  histories  are  discussed  to  enable  administrators 
to  apply  these  theories  to  practical  problems.  Topics  that  are 
discussed  by  Jensen  include  socio-psychological  factors  of  an 
organization,  policy  considerations,  suggestions  and  methods 
for  implementing  change,  and  indicators  of  organizational  effec- 
tiveness.  Regarding  the  latter,  the  author  delineates  optimum 
organizational  conditions  and  demonstrates  the  effects  on 
organizational  indicators  if  such  conditions  are  violated. 
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6)  Kreitlow,   Burton  W.,  and  Teresa  MacNeil.  "The  School  Board  and 

a  Model  for  Educational  Improvement."  Madison,  Wise: 
Center  for  Cognitive  Learning,  University  of  Wisconsin, 
April,  1970. 

"Important  improvements  in  education  usually  require  gross 
adjustments  throughout  the  entire  school  system.  The  school 
board  is  but  one  of  the  agents  of  educational  change.  Admini- 
strators, teachers,  and  citizens  are  also  major  entities  in 
the  process.  The  introduction  of  improvements  in  school  systems 
requires  knowledge  of  how  change  takes  place.  The  authors 
propose  the  Model  for  Educational  Improvement  as  an  instrument 
for  describing  the  process  of  change  in  school  systems.  This 
model  is  a  composite  of  ideas  derived  from  the  findings  of 
researchers  in  the  fields  of  agriculture  and  education  and 
from  the  author's  observations  of  the  change  process  in  five 
Wisconsin  school  districts.  The  model  describes  the  flow  of  the 
change  process  in  a  school  system.   It  allows  for  the  fact  that 
ideas  for  improvement  may  either  be  supplied  to,  demanded  by, 
or  originated  within  the  school  system.   It  underscores  the 
importance  of  the  role  of  interaction  among  all  the  major 
entities  (school  board,  administrators,  teachers,  and  citizens) 
within  the  system  if  there  is  to  be  optimal  commitment  to 
decisions  for  improvement."   (Author) 

7)  Maguire,  Louis  M.   "Feasibility  Study  and  Analysis  of  the 

Administrative  Management."  Philadelphia:   Research  for 
Better  Schools,  April,  1970. 

This  report  describes  a  series  of  recommendations  made  by  Re- 
search for  Better  Schools  to  the  Harrisburg  City  School  District 
(Pennsylvania).  The  recommendations  focus  on  an  administrative 
reorganization  of  the  district.  This  reorganization,  hopefully, 
would  provide  a  framework  within  which  systematic  planning  could 
be  facilitated. 
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4.0  POLITICS,  COMMUNITY,  IMPLEMENTATION  AND  COMMUNICATION 

4.1  Politics  and  Community 

1)  Altshuler,  Alan  A.   "The  Goals  of  Comprehensive  Planning," 

The  City  Planning  Process  A  Political  Analysis,  (Ithaca, 
N.Y.:   Cornell  University  Press,  1965),  pp.  299-332. 

Altshuler  suggests  that  ideal  comprehensive  planning  is  a 
difficult  process  to  effect.   Although  comprehensive  planners 
profess  a  special  knowledge  of  the  public  interest,  their 
stated  goals  are  too  general  to  be  evaluated  by  politicians 
or  the  community.   Furthermore,  the  author  states  that  it 
goals  are  translated  into  attainable  operational  objectives, 
there  is  the  risk  that  these  less  comprehensive  statements  will 
not  reflect  the  interests  of  the  entire  community.   Moreover, 
he  intimates  that  comprehensive  planning  may  be  anti-democratic 
because  it  assumes  that  conflict  of  interest  is  either  minor 
or  completely  reconcilable.   Finally,  Altshuler  questions 
whether  a  comprehensive  planning  approach  is  superior  to  a 
specialized  planning  framework. 

Note:  This  item  refers  to  Chapter  V,  entitled,  "The  Goals  of 
Comprehensive  Planning." 

2)  Andes,  John,  Roe  L.  Johns,  and  Ralph  B.  Kimbrough   Alternative 

Models  for  Organization  and  Administratioj^fJJrj^lchool 
Systems,  WorkiSgTaper  #4,  U.S.  Office  of  Education  Re- 
search Project  #8-0254.  Gainesville,  Fla . :  University 
of  Florida,  February,  1970. 

This  article  focuses  on  two  basic  areas  of  U.S.O.E.  Research 
Project  #8-0254  (University  of  Florida).  This  report  describes 
in  depth  some  existing  models  of  urban  school  organization, 
and  it  also  provides  an  extensive  discussion  of  alternative 
organization  and  administrative  models  for  urban  school  systems. 
Three  bureaucratic  urban  school  organizational  models  (con- 
solidation, centralization  and  decentralization)  are  treated 
and  their  sub-types.   Alternative  models  discussed  include  the 
pluralistic  models  (Federal,  Egaliterian,  PPBS).   Comments 
indicating  some  of  the  justification  and  necessary  con^ions 
for  their  implementation  are  also  provided.   Finally,  the  report 
deals  briefly  with  the  establishment  of  better  community-school 
communication.  A  mechanism  called  the  System  Spanning  Unit 
serves  to  provide  this  communication  interface. 


This  annotation  also  appears  in  section  3.4. 
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3)  Budgeting  for  National  Objectives.   New  York:   Committee  for 

Economic  Development,  January,  1966. 

Since  the  federal  budget  could  be  an  instrument  for  rational 
policy  decisions  and  a  tool  for  effective  management,  this 
report  focuses  on  methods  by  which  these  two  objectives  could 
be  more  fully  realized.   Having  analyzed  the  federal  budgetary 
process  in  detail,  the  Committee  found  these  inadequacies  in 
the  prevailing  system:  little  attention  to  long-range  planning 
and  program  budgeting,  insufficient  focus  on  broad  organiza- 
tional objectives  that  are  operational,  and  ineffective  organi- 
zation of  the  executive  branch  for  dealing  with  problems  of 
resource  allocation  and  operational  efficiency. 

Because  the  budgetary  process  requires  a  close  association 
between  the  executive  and  legislative  branches,  the  Committee's 
recommendations  affect  both  sectors  of  the  government:   the 
federal  budget  should  be  program-oriented  (the  use  of  PPBS, 
cost/benefit  and  cost-effectiveness  is  suggested),  the  Bureau 
of  the  Budget  should  be  strengthened  to  establish  more  effec- 
tive planning  and  evaluative  procedures,  Congress  should  be 
given  a  greater  role  in  evaluating  alternative  uses  of  funds, 
and  the  General  Accounting  Office  should  institute  a  government- 
wide  financial  audit. 

4)  Callahan,  Raymond  E.   Education  and  the  Cult  of  Efficiency. 

Chicago:   University  of  Chicago  Press,  1962. 

The  author  questions  the  validity  of  the  practice  by  school _ 
administrators  of  adopting  business  practices  and  policies  in 
seeking  organizational  efficiency.  Moreover,  he  discusses  the 
political  weakness  and  vulnerability  of  schoolmen  in  comparison 
to  business-industrial  groups.   In  addition,  Callahan  attacks 
the  educational  community  for  capitulating  to  whatever  demands 
are  made  upon  them.   Finally,  he  presents  evidence  showing  that 
such  responses  do  not  necessarily  provide  a  method  of  operation 
that  will  meet  the  needs  of  the  community. 

5)  Gross,  Neal.   "Who  Applies  What  Kind  of  Pressures?"  Who  Runs 

Our  Schools?   (New  York:   John  Wiley  &  Sons,  1958), 
pp.  45-61. 

In  chapter  five,  the  author  defines  pressure  and  details 
descriptions  of  pressures,  sources  of  pressures  and  various 
cross-pressures  that  are  faced  by  school  superintendents  and 
school  board  members  in  Massachusetts.   He  suggests  alterna- 
tive devices  (such  as  the  arousal  of  public  opinion)  that  can 
be  initiated  to  influence  a  superintendent's  position.   Finally, 
he  discusses  some  methods  that  would  ameliorate  the  pressures 
which  benefit  certain  individuals  as  well  as  those  that  in- 
fluence the  school  program  itself. 
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6)  Kushnick,  Louis.   "Race,  Class  and  Power:  the  New  York  Decen- 

tralization Controversy,"  American  Study,  III  (1969), 
pp.  201-219. 

This  article  presents  a  somewhat  one-sided  description  of  the 
school  decentralization  controversy  that  evolved  in  the  Ocean 
Hill-Brownville  section  of  New  York  City.   Kushnick  views  the 
situation  as  an  attempt  by  the  powerless  (blacks  and  Puerto 
Ricans)  to  gain  power  from  an  unresponsive,  centralized  edu- 
cational bureaucracy.   He  traces  the  arguments  of  the  proponents 
of  decentralization  and  shows  how  their  efforts  met  stern, 
negative  responses  from  both  the  Board  of  Education  and  the 
United  Federation  of  Teachers. 

7)  Levin,  Peter,  and  David  Donnison.   "People  and  Planning,"  Public 

Administration,  XLVII  (Winter,  1969),  pp.  473-479. 

This  article  analyzes  the  Skeffington  Committee  Report  (England), 
an  inquiry  into  public  participation  in  town  planning.  The 
authors  describe  the  changing  social  atmosphere  under  which  the 
report  was  released.   In  addition,  Levin  and  Donnison  list 
some  basic  recommendations  of  the  report.  Finally,  they  focus 
on  some  weaknesses  inherent  in  the  participation  of  the  public 
in  town  planning. 

8)  Sandberg,  John  (ed.)  Schools  and  Comprehensive  Urban  Planning. 

Portland,  Oreg. :  Northwest  Regional  Educational  Laboratory, 
September,  1969. 

The  purpose  of  this  document  is  to  suggest  ideas  that  will  help 
school  administrators  work  more  effectively  with  local,  state 
and  regional  planning  bodies.  The  authors  stress  the  urgency 
for  comprehensive,  interagency  planning  in  urban  areas.  A 
series  of  articles  examine  some  aspects  of  integrative,  cooper- 
ative planning.   For  example,  John  Sandberg' s  article  assesses 
some  of  the  problems  that  beset  agencies  that  try  integrative 
planning.   In  an  article  entitled,  "Urban  Planning  Objectives,' 
the  authors  list  certain  "do's  and  don'ts"  for  involving 
citizens  in  the  planning  process.  They  also  state  that  the 
quality  of  public  participation  depends  largely  on  the  people  s 
access  to  facts  and  ideas.  The  latter  two  articles  by  Hanson 
and  Musolf  discuss  planning  models  (types  of  and  strategies 
used  to  bring  about  change)  and  the  stages  in  the  development 
of  an  information  system  for  urban  areas. 

This  annotation  also  appears  in  sections  2.1  and  3.2. 
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9)  Tye,  Kenneth  A.   "A  Framework  for  Analyzing  Relationships 

Between  Public  Demands  and  School  Board  Decisions."  Paper 
presented  at  the  Annual  Meeting  of  the  American  Educational 
Research  Association,  Minneapolis,  March,  1970. 

This  report  describes  actual  interactions  between  the  public 
and  its  school  board.  The  author  refers  to  a  structural- 
functional  systems  framework  devised  by  David  Easton  and  Gabriel 
Almond  to  show  how  political  science  theory  applies  to  educa- 
tional decision-making  units.  Tye  describes  this  framework, 
applies  it  to  data  from  a  large  urban  school  district  and    _ 
offers  some  implications  and  guidelines  for  future  research  in 
assessing  the  relationship  between  the  school  board  and  its 
environment. 

10)  Vanecko,  James  J.,  et  al.   Community  Organization  Efforts, 

Political  and  Institutional  Change,  and  the  Diffusion  of 
Change  Produced  by  Community  Action  Programs.   Clearmg- 
house  for  Federal  Scientific  and  Technical  Information 
Contract  #PB188692.   Chicago:  University  of  Chicago, 
December,  1969. 

The  purpose  of  this  report  is  to  assess  characteristics  which 
determine  the  effectiveness  of  Community  Action  Agencies  (CAAsj 
in  influencing  other  institutions  to  become  more  responsive  to 
the  needs  and  demands  of  the  poor.  The  institutions  included 
were  public  school  systems,  private  welfare,  employment,  and 
community-political  agencies.  Response  changes  from  these _ 
institutions  were  measured  using  gamma  correlation  coefficients 
to  determine  if  the  CAA's  had  had  significant  influence,  h 
full  explanation  of  the  statistical  measures  are  included  in 
the  report. 

The  CAA's  were  organized  into  three  distinct  goal  patterns: 
social  service  and  educational  orientation  (tradition),  community 
organization  and  mobilization  orientation,  and  employment 
orientation.   It  was  found  that  community  organization  and 
mobilization  structure  consistently  showed  the  strongest  assoc- 
iation with  institutional  change.  Regarding  the  public  schools, 
when  CAA  community  organization  emphasis  is  processed  through 
neighborhood  activity,  residents  are  mobilized  to  work  with 
the  PTA  and  other  community  groups  to  press  for  change  and 
thus  strengthen  already  present  citizen  pressure  activity.  When 
this  citizen  pressure  is  further  activitated,  change  is  more 
likely  to  occur. 
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11)  Wildavsky,  Aaron.  The  Politics  of  the  Budgetary  Process. 

Boston:  Little,  Brown  and  Co.,  Inc.,  1964. 

The  purpose  of  this  book  is  to  describe  and  evaluate  the 
budgetary  process.  The  first  chapters  of  the  book  are  devoted 
to  the  kinds  of  calculations  and  types  of  strategies  that  are 
used  by  participants  in  the  budgetary  process  to  accomplish 
their  purposes.  Budgeting  is  characterized  as  being  incre- 
mental, sequential,  fragmentary,  etc.  and  not  possessing  rational 
techniques  such  as  cost/benefit  analysis  and  PPBS.   In  fact, 
Wildavsky  later  claims  that  PPBS  represents  concepts  that  are 
completely  contrary  to  the  concepts  of  traditional  budgeting. 
Furthermore,  he  states  that  budgetary  reforms  should  be  aimed 
at  the  political  system  which  the  budget  expresses.   Reforms 
that  alter  only  the  budget  instrument  are  insufficient.   In 
the  concluding  sections  of  the  book,  Wildavsky  proposes  that 
the  present  budgetary  system  works  far  better  than  is  thought 
and  he  examines  this  hypothesis  through  concepts  such  as  com- 
prehensiveness and  coordination  that  occur  in  present  day 
budgeting.  The  appendix  offers  the  reader  a  detailed  descrip- 
tion of  the  budgetary  process. 

12)  Wildavsky,  Aaron.   "The  Political  Economy  of  Efficiency," 

The  Public  Interest  (Summer  1967),  pp.  30-48. 

Wildavsky  proposes  a  policy  analysis  technique  as  an  alterna- 
tive to  PPBS.   He  analyzes  and  compares  experiences  of  the 
Defense  Department  and  domestic  agencies  with  PPBS.   He  con- 
cludes that  PPBS  has  been  damaging  to  prospects  for  improved 
policy  analysis  at  the  federal  level.   Finally,  he  suggests 
ways  to  facilitate  better  policy  analysis. 

This  annotation  also  appears  in  section  2.2. 

13)  Wildavsky,  Aaron.   "Rescuing  Policy  Analysis  form  PPBS,"  Public 

Administration  Review  (March-April,  1969),  pp.  189-202. 

Wildavsky  examines  economic  efficiency  techniques  (cost/benefit 
analysis,  systems  analysis,  and  program  budgeting)  in  order  to 
assess  whether  or  not  these  methods  achieve  efficiency,  the 
author  suggests  that  efficiency  is  a  function  of  the  political 
system  as  well  as  of  economizing  within  the  system.   He  con- 
cludes that  both  cost/benefit  analysis  and  systems  analysis 
promote  "mixed"  efficiency  while  minimally  modifying  the ^ 
political  system.  On  the  other  hand,  program  budgeting  is  a 
function  of  policy  politics.  Wildavsky  concludes  that  economic 
efficiency  techniques  must  not  overlook  the  affects  of  political 
costs  and  benefits  of  change. 
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14)  Wilder,  David  E.,  et  al.  Actual  and  Perceived  Consensus  on 

Educational  Goals  Between  School  and  Community.   U.S.  Office 
of  Education  Project  #5-1067-2-12-1.   New  York:  Columbia 
University,  December,  1968. 

This  report  discusses  the  goals  of  education  as  seen  by  mothers 
of  students,  teachers,  and  students.  The  goals  used  in  this 
study  were  excerpted  from  Downey's  Tasks  of  Public  Education 
(see  section  1.2).  The  essential  purpose  of  this  report  is 
to  assess  the  effects  that  actual  and  perceived  differences 
in  goals  between  the  participants  have  on  the  social  environ- 
ment of  the  school.  Although  consensus  about  educational  goals 
among  mothers,  teachers,  and  students  is  not  significantly  high, 
the  authors  list  and  describe  several  reasons  for  the  continued 
survival  of  schools. 

4.2   Implementation 

1)  A  Program  to  Introduce  Improved  Management  Methods  in  Hawaii 

state  Government:  A  Planning-Budgeting  System.  A  Review 

and  Evaluation  of  the  First  Year  of  Implementation, 
Central  Analysis  Group,  Office  of  the  Governor,  Clearing- 
house for  Federal  Scientific  and  Technical  Information  Pro- 
ject #P-20.  Springfield,  Va.:   Department  of  Planning  and 
Economic  Development,  March,  1969. 

The  purpose  of  this  effort  was  to  improve  the  effectiveness  of 
state  planning  and  to  assure  that  program  plans  become  the 
basis  for  program  decisions.  Volume  I  forms  the  basic  framework 
for  a  four  volume  report.   A  detailed  description  is  provided 
of  this  task-oriented  endeavor.  The  planning  approach  was  _ 
designed  to  permit  an  evolution  from  descriptive  and  histori- 
cally-based reports  to  those  which  were  both  analytical  and 
forward-looking.   Decentralized  program  planning  allowed  PPBS 
to  be  regarded  as  a  logical  outgrowth  of  existing  trends  rather 
than  as  a  completely  new  technique  which  might  have  negatively 
affected  the  employee  climate. 

2)  Benton,  J.B.,  and  A.J.  Tenzer.   "Program  Budgeting  and  Executive 
Commitment."   Santa  Monica,  Calif.:  Rand  Corporation, 
July,  1969. 

The  authors'  intention  is  to  deal  with  issues  that  arise  when 
attempts  are  made  to  implement  program  budgeting  within  solidly 
entrenched  bureaucracies.  The  paper  is  not  about  program 
budgeting  concepts,  but  rather  it  concerns  strategies  that  can 
be  adopted  to  bring  about  organizational  change  required  for 
program  budgeting.   In  describing  the  problems  of  implementation, 
the  authors  cite  the  case  of  the  New  York  City  Police  Department 
as  an  example.  They  devote  considerable  attention  to  the  need 
for  an  evolutionary  process  of  implementation  and  the  necessity 
for  multiple  channels  of  communication. 

This  annotation  also  appears  in  section  4.3. 
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3)  Cook,  Desmond  L.   "A  Generalized  Project  Management  System 

Model."  Columbus,  Ohio:  Ohio  State  University,  Educa- 
tional Program  Management  Center,  November,  1968. 

The  author  presents  a  generalized  project  management  model  in- 
tended for  the  management  of  educational  projects.   He  identi- 
fies the  two  major  components  of  the  project  management  system 
as  the  planning  subsystem  and  the  control  subsystem.   Each 
sub-function  within  the  two  components  of  the  model  is  de- 
scribed in  terms  of  desired  inputs,  processes,  and  outputs. 
The  generalized  nature  of  the  model  enables  it  to  be  adapted 
to  project  situations. 

4)  Croston,  J.D.,  and  G.  Gregory.   "A  Critique  of  Operational 

Research  and  Decision-making  by  Adelson  and  Norman," 
Operational  Research  Quarterly,  XX  (December,  1969), 
pp.  415-420. 

The  authors  discuss  and  criticize  a  particular  operational 
research  and  decision-making  model  and  some  if  its  implications. 
Discussions  focus  on  involving  many  people  within  an  organi- 
zation in  planning  and  decision-making. 

One  of  the  significant  points  in  the  paper  is  a  distinction 

made  between  the  usual  "vertical"  information  flow  within  an 

organization  and  the  "horizontal"  information  flow  needed  to 
plan  by  objectives. 

This  annotation  also  appears  in  section  4.3. 

5)  Curtis,  William  H.   "Program  Budgeting  Design  for  Schools 

Unveiled,  with  Much  Work  Still  To  Go,"  Nation's  Schools, 
LXXXIV  (November,  1969),  pp.  40-43. 

This  article  presents  an  interview  with  William  H.  Crutis, 
project  director  for  the  Research  Corporation  of  the  Associa- 
tion of  School  Officials,  an  agency  which  has  undertaken  a 
three-year  project  to  develop  a  "program  planning  budgeting 
evaluation  system  (PPBES)  for  local  schools."  This  interview 
assesses  some  of  the  problems  and  advances  of  PPBES  in  the 
established  pilot  projects. 

6)  Eberle,  Robert  F.   "Personnel  Management  for  Change  and  Innova- 

tion in  Education,"  Journal  of  Creative  Behavior,  III 
(Fall,  1969),  pp.  277-283. 

Eberle  demonstrates  that  involving  individuals  in  decision-making 
that  affects  them  is  a  source  of  motivation.   Moreover,  the 
manager  who  promotes  self-confidence  and  faith  in  employees 
contributes  to  the  success  of  a  change  effort.  The  author  also 
suggests  that  managers  study  group  dynamics  to  learn  how  group 
membership  influences  change.   Finally,  Eberle  states  that 
change  should  be  initiated  by  school  administrators;  change 
should  not  come  about  solely  as  a  reaction  to  demands  or 
pressures. 
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7)  Foster,  Charles  W.  (ed.)  The  First  National  Conference  on  PPBES 

in  Education.  Chicago:  Research  Corporation  of  the  Associa- 
tion of  School  Business  Officials,  1969. 

The  Research  Corporation  of  the  American  Association  of  School 
Business  Officials  has  been  working  on  a  PPBS  design  under  a 
USOE  multi-year  grant.   Field  efforts  are  underway  in  Dade 
County,  Florida,  and  eight  pilot  districts  throughout  the 
country. 

This  document  includes  a  history  of  program  budgeting  by  David 
Novick  of  the  RAND  Corporation  and  a  joint  report  by  William 
Curtis,  project  director  and  John  Gott,  research  associate  for 
the  project.  The  joint  presentation  describes  some  of  the 
experience  and  new  thinking  resulting  from  the  previous  year's 
field  experience. 

8)  Hartley,  Harry  J.   "PPBS:  Economic  Stategy  for  Urban  Educa- 

tion," Urban  Review,  III  (1968),  pp.  27-29. 

The  author  examines  PPBS  at  the  conceptual-descriptive  level 
indicating  the  major  differences  between  this  method  and  the 
traditional  budgetary  process.   Hartley  reports  on  the  progress 
of  those  agencies  which  have  adopted  PPBS  on  a  partial  scale 
of  operations.   The  author  suggests  that  two  values  result  from 
the  implementation  of  PPBS:  educators  reconsider  their  pri- 
orities based  upon  program  design,  and  the  gap  that  exists 
between  professed  and  observed  values  based  upon  available 
resources  is  displayed. 

9)  Horvat,  John  J.   "Content  and  Strategies  of  Communication  in 

Current  Educational  Change  Efforts."  A  Paper  prepared  for  the 
American  Association  of  School  Administrators  Educational 
Press  Association  discussion  group  on  Communication 
Strategies  of  Educational  Change,  February,  1967. 

In  this  report,  Horvat  examines  the  content  of  educational 
change  communication  and  the  communication  strategies  that  are 
being  employed.  These  two  factors  form  the  basis  of  a  two- 
dimensional  schema  in  which  the  author  evaluates  the  effective- 
ness of  certain  communication  devices  to  types  of  communication 
content.   He  describes  five  kinds  of  communication  content  that 
he  has  observed  and  he  assesses  the  strengths  and  weaknesses 
of  five  common  communication  media  (strategies). 
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10)  Innovations  in  Planning,  Programming,  and  Budgeting  in  State 

and  Local  Governments.  A  Compendium  of  Papers  submitted 
to  the  Subcommittee  on  Economy  in  Government  of  the  Joint 
Economic  Commitree  Congress  of  the  United  States.  Washing- 
ton, D.C. :   August,  1969. 

This  reference  is  composed  of  a  series  of  articles  each  of 
which  describes  experience  with  establishing  a  PPB  System  at 
the  federal,  state  or  county  level.  The  articles  are  case 
studies  dealing  with  PPBS  operations  of  the  national  govern- 
ment, California,  Michigan,  New  York  State,  Wisconsin,  Dade 
County  (Florida).  Los  Angeles  County,  (California)  Metropolitan 
Nashville-Davidson  County  (Tennessee),  Nassau  County  (New  York), 
Wayne  County  (Michigan),  and  Dayton,  Ohio.   Included  in  the  case 
studies  are  various  descriptions  of  the  basic  concepts  of  PPBS, 
associated  program  structures  and  uses  of  the  analytical  tools. 

11)  Knezevich5  S.J.   "Symposium  on  Organization  and  Resource  Alloca- 

tions Required  by  School  Systems  Desiring  to  Implement 
PPBS."  Washington,  D.C:  AASA  National  Academy  for  School 
Executives,  n.d. 

The  author  recommends  a  gradual  introduction  to  implement  PPBS. 
He  also  suggests  that  the  quantity  and  quality  of  administrative 
staff  be  increased.  The  author's  recent  leadership  role  with 
the  American  Association  of  School  administrator's  Academy  for 
School  Executives  suggests  that  training  may  provide  a  partial 
answer  to  this  "quantity-quality"  gap. 

12)  Langenbach,  Michael.   "Development  of  an  Instrument  to  Measure 

Teachers'  Attitudes  Toward  Curriculum  Use  and  Planning. 
Norman,  Okla.:  University  of  Oklahoma,  1969. 

The  purpose  of  this  study  was  to  construct  an  instrument  that  _ 
would  measure  the  relationship  between  teacher  participation  in 
curriculum  use  and  planning  and  positive  attitude  towards  school 
administration.  Teachers  were  selected  as  study  subjects  based 
upon  their  preconceived  ideas  towards  curriculum  use  and  whether 
or  not  they  were  ever  involved  in  a  curriculum  planning  program. 
After  the  respondents'  information  was  tabulated,  certain  null 
hypotheses  were  tested.  Study  conclusions  substantiate  the 
notion  that  teachers'  attitudes  are  positively  affected  by 
participation  in  a  program  of  curriculum  use  and  planning.  Grade 
assignment  and  years  of  experience  are  two  other  factors  to 
consider. 
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13)  Maguire,  Louis  M.  "Change  Literature  and  the  Practicing  School 

Administrator."  Paper  presented  at  a  Meeting  for  the 
American  Educational  Research  Association  Symposium,  Minnea- 
polis, March,  1970. 

The  author's  premise  is  that  there  is  little  practical  help  from 
the  literature  for  the  school  administrator  in  dealing  with 
problems  of  change.  Maguire  lists  12  underlying  causes  which 
support  his  major  premise.   Implicit  in  this  presentation  is 
the  point  that  communication  problems  between  theoretician  and 
user  are  fundamental  and  salient. 

14)  Temkin,  Sanford.   "Comprehensive  Planning  for  School  Districts." 

Paper  prepared  for  American  Educational  Research  Association 
Symposium,  Minneapolis,  March,  1970. 

The  author  feels  that  current  "PPBS  approaches"  present  serious 
implementation  problems  for  school  districts.   He  provides  a 
methodology  for  engaging  in  comprehensive,  system-wide  planning 
for  school  districts.  The  author  contends  that  an  evolutionary 
installation  of  the  planning  mechanism  is  required  because  the 
planning  mechanism  itself  causes  a  great  deal  of  change. 

Once  the  planning  mechanism  has  been  installed  information  flows 
can  be  generated  to  evaluate  the  system  as  it  is  in  order  to 
form  a  basis  for  future  change  decisions. 

This  annotation  also  appears  in  section  2.1. 

4.3  Communication 

1)  Benton,  J.B.,  and  A.J.  Tenzer.   "Program  Budgeting  and  Execu- 
tive Commitment."  Santa  Monica,  Calif.:  Rand  Corporation, 
July,  1969. 

The  authors'  intention  is  to  deal  with  issues  that  arise  when 
attempts  are  made  to  implement  program  budgeting  within  solidly 
entrenched  bureaucracies.  The  paper  is  not  about  program 
budgeting  concepts,  but  rather  it  concerns  strategies  that  can 
be  adopted  to  bring  about  organizational  change  required  for 
program  budgeting.   In  describing  the  problems  of  implementation, 
the  authors  cite  the  case  of  the  New  York  City  Police  Department 
as  an  example.  They  devote  considerable  attention  to  the  need 
for  an  evolutionary  process  of  implementation  and  the  necessity 
of  multiple  channels  of  communication. 

This  annotation  also  appears  in  section  4.2. 
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2)  Croston,  J.D.,  and  G.  Gregory.   "A  Critique  of  Operational 

Research  and  Decision-making  by  Adelson  and  Norman," 
Operational  Research  Quarterly,  XX  (December,  1969), 
pp.  415-420. 

The  authors  discuss  and  criticize  a  particular  operational 
research  and  decision-making  model  and  some  of  its  implications, 
Discussions  dwell  on  involving  many  people  within  an  organi- 
zation in  planning  and  decision-making. 

One  of  the  significant  points  in  the  paper  is  a  distinction 

made  between  the  usual  "vertical"  information  flow  within  an 

organization  and  the  "horizontal"  information  flow  needed  to 
plan  by  objectives. 

This  annotation  also  appears  in  section  4.2. 

3)  Horvat,  John  J.   "Content  and  Strategies  of  Communication  in 

Current  Educational  Change  Efforts."  A  Paper  prepared  for 
the  American  Association  of  School  Administrators,  Educa- 
tional Press  Association  discussion  group  on  Communication 
Strategies  of  Educational  Change,  February,  1967. 

In  this  report,  Horvat  examines  the  content  of  educational 
change  communication  and  the  communication  strategies  that 
are  being  employed.  These  two  factors  form  the  basis  of  a 
two-dimensional  schema  in  which  the  author  evaluates  the 
effectiveness  of  certain  communication  devices  to  types  of 
communication  content.   He  describes  five  kinds  of  communica- 
tion content  that  he  has  observed  and  he  assesses  the  strenghs 
and  weaknesses  of  five  common  communication  media  (strategies). 

This  annotation  also  appears  in  section  4.2. 

4)  Shannon,  C.E.,  and  W.  Weaver.  The  Mathematical  Theory  of 

Communication.   Urbana :  University  of  Illinois  Press,  1949. 

This  book  presents  a  mathematical  theory  of  communication  based 
on  a  concept  of  entropy  from  a  physical  law  of  thermodynamics. 

One  of  the  ideas  which  is  treated  deals  with  communication  as 
a  function  of  organization. 
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5.0  BIBLIOGRAPHY 


1)  Blue,  Richard,  et  al.   Bibliography  on  Planned  Social  Change 

(With  Special  Reference  to  Rural  Development  and  Educational 
Development).  Volumes  I,  II,  III.  Government  Reports, 
U.  N.  Reports  and  Proceedings  of  Special  Conferences.   U.S. 
Department  of  Commerce.  Minneapolis:  University  of  Minne- 
sota, January,  1967. 

This  annotated  bibliography  was  prepared  in  conjunction  with  the 
research  activities  of  the  CIC-AID  Rural  Development  Research 
Project.  The  bibliography  covers  a  wide  range  of  socio-economic 
research  endeavors.   Volume  I  includes  periodical  materials 
covered,  Volume  II  deals  with  books  and  book-length  monographs, 
and  Volume  III  covers  government  and  United  Nations  publications 
and  proceedings  of  specialized  conferences. 

2)  Bowman,  Mary  Jean.   "Economics  of  Education,"  Review  of  Educa- 

tional Research,  XXXIX,  (Washington,  D.C. :  American 
Educational  Research  Association,  1969),  pp.  641-670. 

This  review  of  the  literature  focuses  on  selected  aspects  of 
macro-economic  analysis.  Among  these  are  human  resource  and 
human  capital  formation,  national- income  accounting,  aggrega- 
tive production  functions,  and  a  related  economic  problem 
currently  being  labeled  as  the  "economics  of  education." 

A  bibliography  containing  42  entries  is  provided. 

3)  Chartrand,  Robert  L.,  and  Louise  G.  Becker.  PPBS  in  1970: 

Methodology  and  Implementation.  Washington,  D.C:  Library 
of  Congress,  April,  1970. 

This  annotated  bibliography  presents  sources  of  information 
which  set  forth  the  methodology  of  PPBS.   Included  among  the 
20  articles  are  governmental  directives,  congressional  hearings, 
symposia  proceedings,  reports  on  governmental  experience  and 
other  background  material.  Finally,  the  bibliography  also  lists 
28  other  selections  which  are  not  annotated. 

4)  Directory  of  Organizations  and  Personnel  in  Educational  Admini- 

stration.   Eugene,  Oreg. :   ERIC  Clearinghouse  on  Educat io na 1 
Administration,  1962. 

This  directory  is  divided  into  two  sections.  The  first  section 
is  a  listing  of  organizations  that  includes  their  areas  of 
specialization,  publications,  and  policies  for  supplying  informa- 
tion to  inquirers.  The  second  section  is  a  listing  of  personnel 
who  are  involved  in  educational  administration  research. 
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5)  Educational  Goals  and  Objectives  in  California.  Sacramento, 
Calif. :  California  School  Boards  Association,  August, 
1969. 

This  report  consists  of  the  goals  and  objectives  formulated  by 
900  school  districts  of  California  in  conjunction  with  the 
California  School  Boards  Association.  Distinctions  among 
philosophies,  goals,  and  objectives  are  described.  Eighteen 
general  goal  categories  have  been  generated.  Also,  51  entries 
appear  in  a  bibliography  on  goals  and  objectives. 

This  annotation  also  appears  in  section  1.2. 

6)  ERIC  Abstracts:  A  Collection  of  ERIC  Document  Resumes  on 

Program  Budgeting  and  Cost  Analysis.   Series  Number  Five. 
Eugene,  Oreg. :  University  of  Oregon,  American  Association 
of  School  Administrators,  January,  1970. 

This  annotated  bibliography  presents  51  entries  dealing  with 
problems  of  economic  efficiency  particularly  in  the  field  of 
education.  The  efficiency  techniques  mentioned  include 
PPBS,  cost-effectiveness,  cost/benefit. 

7)  Fishburn,  Peter  C.   ''Utility  Theory,"  Management  Science, 

XIV  (January,  1968),  pp.  335-378. 

Approximately  300  entries  surveying  utility  theory  are  found 
in  the  writer's  bibliography. 

(See  also  citation  in  section  1.3). 

8)  Goals  for  Public  Education  in  Texas.  A  Report  by  the  Sub- 

committee on  Goals  to  the  Governor's  Committee  on  Public 
School  Education,  U.S.  Office  of  Education.  Texas: 
Operation  PEP,  December,  1968. 

This  report  traces  the  historical  evolution  of  educational 
goals.   It  is  intended  as  an  aid  to  those  educational  planners 
and  managers  who  must  translate  educational  objectives  into 
operational  terms.  A  bibliography  of  consulted  references  is 
attached  to  the  study  as  well  a  : ixes  which  contain 

goal  statements  from  quoted  sources. 

This  annotation  also  appears  in  section  1.2. 

9)  Judy,  Richard  W.   "Systems  Analysis  for  Efficient  Resource 

Allocation  in  Higher  Education."  Canada:  University  of 
Toronto,  Institute  for  the  Quantitative  Analysis  of  Social 
and  Economic  Policy,  January,  1970. 

A  bibliography  providing  listings  of  32  articles  and  r€ 
available  from  the  University  of  Toronto  is  includ 
Judy  document  annotated  in  sectior. 
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10)  Maguire,  Robert  F.   "Planning  Approaches  in  the  Public  Sector." 

Philadelphia:  Research  for  Better  Schools,  June,  1969. 

Maguire  surveys  various  efficiency  techniques  and  approaches 
used  in  pubiic  sector  planning  (particularly  in  transportation, 
water  resources,  and  social  services).  Consideration  is  given 
to  PPBS,  cost/benefit  analysis,  cost-effectiveness  analysis, 
etc.  in  evaluating  their  inherent  strengths  and  weaknesses 
for  public  sector  decision-making. 

11)  McAbee,  Harold  V.  The  Oregon  Planning  Programming  Budgeting 

Systems  Institute.   Monmouth,  Oreg. :  Teaching  Research 
Division,  Oregon  State  System  of  Higher  Education,  December, 
1968. 

A  section  of  the  final  report  (pages  78-84)  consists  of  a  series 
of  approximately  100  articles,  reports,  and  program  memoranda 
concerning  topics  such  as  PPBS,  systems  analysis,  cost/benefit 
and  cost-effectiveness  analysis. 

(See  also  citation  in  section  2.2). 

12)  McGivney,  Joseph  H.,  and  William  C.  Nelson.  Program,  Planning, 

Budgeting  Systems  for  Educators.   Volume  III:  An  Annotated 
Bibliography.   Columbus,  Ohio:  Ohio  State  University, 
December,  1969.   (ERICft  ED  035756) 

"This  annotated  bibliography  is  composed  of  70  citations  rele- 
vant for  PPBS  and  ranging  in  date  from  1962  to  1969  but 
emphasizing  the  1965  to  1968  period.   Listings  are  arranged 
alphabetically  according  to  author  under  the  general  divisions: 
Overview  of  PPBS,  Systems  and  Administrative  Theory,  The 
Planning  Process,  Program  Budgeting  and  Management  Control, 
Identification  and  Analysis  of  Investment  alternatives,  Educa- 
tion and  Economics,  Data  and  Statistical  Analysis,  and  Limita- 
tions of  PPBS.  An  author  index  is  included."  (Author) 

(See  also  citation  in  section  2.2). 
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13)  Piele,  Philip  K. ,  and  David  G.  Bunting.  Program  Budgeting  for 

the  School  Administrator:  A  Review  of  Dissertations  and 
Annotated  Bibliography.   Eugene,  Oreg. :   ERIC  Clearing- 
house on  Educational  Administration,  September,  1969. 

In  this  booklet,  the  authors  have  synthesized  many  significant 
ideas  of  program  budgeting  which  have  arisen  from  doctoral 
dissertations.  Their  purpose  is  to  disseminate  this  informa- 
tion to  school  administrators  who  are  considering  the  possi- 
bility of  adopting  a  program  budget.  Finally,  the  authors 
provide  an  annotated  bibliography  of  the  dissertations  used  in 
their  presentation. 

This  annotation  also  appears  in  section  2.2. 

14)  Research  for  Better  Schools.  An  Annotated  Bibliography  of 

Benefits  and  Costs  in  the  Public  Sector.   (Philadelphia: 
November,  1968. 

This  annotated  bibliography  presents  nearly  3,000  entries 
dealing  with  problems  of  economic  efficiency  in  the  public 
sector.   Entries  deal  with  techniques  such  as  benefit/cost 
analysis,  cost-effectiveness  analysis,  operational  analysis, 
PPBS,  etc.  Particular  emphasis  is  directed  to  the  field  of 
education. 

15)  Richard,  Robert.   Subjective  Social  Indicators.   Clearinghouse 

for  Federal  Scientific  and  Technical  Information,  Contract 
#PB187944.   Chicago:   National  Opinion  Research  Center, 
September,  1969. 

An  annotated  bibliography  intended  for  a  survey  sponsor  or  a 
study  director  is  included  (after  Chapter  IV)  as  well  as  a 
total  bibliography  of  sources  used  (pp.  185-189.). 

(See  also  citation  in  section  1.4). 

16)  Stimson,  David  H.   "Utility  Measurement  in  Public  Health 

Decision  Making,"  Management  Science,  XVI  (October,  1969), 
pp.  B-17-B-30. 

A  bibliography  listing  44  items  relating  to  utility  and  prefer- 
ence measurement  is  provided  at  the  end  of  the  Stimson  article 
cited  in  section  1.3. 
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4.2.12 

4.2.13 

4.3.1 

5.4 

5.8 

5.12 

5.13 

5.14 

Alternatives 

1.3.4 

1.3.13 

1.4.7 

1.4.10 

1.5.5 

2.2.8 

2.3.5 

2.3.6 

2.3.8 

2.3.10 

2.3.13 

2.4.2 

2.4.8 

2.6.1 

2.6.4 

3.1.1 

3.2.2 

3.4.1 

4.1.2 

4.1.3 

Bayesian 

analysis 

1.3.10 

1.3.21 

2.6.1 

Budgeting 

1.3.2  1.3.14  1.3.15  1.5.2 
1.5.4  2.2.3  2.2.4  2.2.6 
2.4.4  2.4.7  2.6.1  3.2.2 

4.1.3  4.1.11  4.2.8  5.6 
5.14 

California 

1.2.3   4.2.10  5.5     5.14 


Change  of  educational  systems 

2.1.2  2.1.4  3.2.6  3.4.5 

3.4.6  4.1.6  4.1.8  4.1.10 

4.1.13  4.2.6  4.2.9  4.2.13 

4.2.14  4.3.3 


Choice,  theory  of 
2.3.15 

Communication 

3.3.1   4.2.9  4.3.3   4.3.4 

Comprehensive  planning 

1.5.1  2.1.3  2.1.4  2.1.5 
2.3.1  2.4.6  3.2.1  3.2.6 
4.1.1   4.1.8   4.2.14 

Conflict  resolution 
1.3.16     3.4.4 

Cost/benefit  analysis 
See  Costs  and  benefits 

Cost-effectiveness  analysis 

1.5.3  1.5.4  2.1.1  2.1.5 

2.2.3  2.3.15  2.4.1  2.4.2 

2.4.3  2.4.5  2.4.6  2.4.7 

2.4.8  4.1.3  5.6  5.10 

5.11  5.14 


Cardinal  utility 
1.3.4 
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Costs  and  benefits 


Educational  objectives 


1.4.4 

1.4.6 

1.4.13 

1.5.1 

2.1.5 

2.2.3 

2.2.4 

2.2.7 

2.3.5 

2.3.6 

2.3.7 

2.4.6 

2.6.1 

2.6.2 

2.6.3 

2.6.4 

3.1.3   4.1.3   4.1.11  4.1.13 


5.6 


5.10 


5.11 


5.14 


Counties 

1.3.3   4.2.10 


Critical   incident  technique 
1.1.1 


Decision-making 


1.3.2 

1.3.8 

1.3.10 

1.3.13 

1.3.18 

1.3.19 

1.3.21 

1.4.1 

1.4.13 

1.5.2 

2.1.2 

2.2.4 

2.2.7 

2.3.2 

2.3.8 

2.3.12 

2.3.13 

2.3.15 

2.4.6 

2.5.1 

2.6.1 

2.6.2 

2.6.3 

3.1.2 

3.2.3 

3.2.4 

3.4.2 

4.1.3 

4.2.4 

4.2.6 

4.3.2 

5.10 

5.16 

Delphi  method 

1.3.3   1.3.6   1.3.11  1.3.16 

Denver 

1.5.1   3.2.1 

Educational  goals 

See  Educational  objectives 

Educational  needs  assessment 

1.1.1   1.1.3   1.1.4   1.1.6 
1.2.1   1.4.1 


1.1.4 

1.1.5 

1.1.6 

1.2.1 

1.2.2 

1.2.3 

1.2.4 

1.2.5 

1.3.3 

1.3.5 

1.3.7 

1.4.1 

1.4.2 

1.4.5 

1.4.13 

1.5.4 

2.1.3 

2.2.1 

2.2.5 

2.2.7 

2.3.10  2.5.2   3.2.6   3.4.2 


3.4.4   4.1.14  5.5 


Educational  planning" 


Educational  policy 


5.8 


1.2.4 

1.3.2 

1.3.14 

1.3. IS 

1.4.1 

1.4.11 

1.4.13 

1.5.1 

1.5.4 

2.1.3 

2.1.4 

2.1.5 

2.2.2 

2.2.5 

2.2.7 

2.3.5 

2.3.10 

2.3.11 

2.4.5 

2.4.6 

3.1.1 

3.4.4 

3.4.7 

4.1.3 

4.1.7 

4.2.1 

4.2.3 

4.2.4 

4.2.12 

4.3.2 

5.10 

5.14 

1.1.6 

1.3.2 

1.4.2 

2.3.13 

3.2.3 

3.4.4 

4.1.3 

4.1.5 

4.1.11 

Educational  preferences 

1.1.4   1.3.9   1.3.13   5.16 

Educational  priorities 

1.1.4  1.2.2        1.3.2        1.3.3 

1.3.5  1.3.14     1.3.19     1.4.2 
2.3.10     2.4.4       4.2.8 

Educational  problems 
1.1.1   1.2.1   1.3.2 

Education,  economics  of 
2.3.13  5.2 
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Effectiveness 

See  Measurement  of  effectiveness 
techniques 

Florida 

3.2.7   4.2.7   4.2.10 

Georgia 
2.3.1 

Gradient  method 
2.4.8 

Harrisburg 
3.4.7 

Hawaii 
4.2.1 

Information  systems 

1.3.2  1.5.1  2.1.3  2.2.1 
3.2.1  3.2.3  3.2.4  3.2.5 
3.2.6   3.2.7   4.1.8   5.1.4 

Input-output  theory 

1.4.11  2.1.2  2.5.1  2.5.4 
2.5.5 

Kentucky 
2.5.3 

Louisville 
1.3.18 


Macro-economic  analysis 
5.2 

Management 

See  Administration 

Management  information  systems 
See  Information  systems 

Massachusetts 

4.1.5 

Maximization 
2.3.3 

Measurement  of  effectiveness 
techniques 


1.1.1 

1.1.5 

1.2.2 

1.3.1 

1.3.8 

1.3.12 

1.3.14 

1.3.15 

1.3.17 

1.3.18 

1.3.19 

1.3.21 

1.4.2 

1.4.3 

1.4.4 

1.4.5 

1.4.6 

1.4.7 

1.4.8 

1.4.10 

1.4.11 

1.4.12 

1.4.14 

1.5.3 

2.1.1 

2.2.5 

2.3.5 

2.3.6 

2.3.7 

2.3.8 

2.3.9 

2-3.13 

2.3.14 

2.4.5 

2.4.8 

2.5.2 

2.5.4 

2.5.5 

3.1.1 

3.1.2 

5.14 

Michigan 

4.2.10 

Models 


1.1.6 

1.3.1 

1.3.12 

1.3.13 

1.3.14 

1.3.18 

1.3.19 

1.4.4 

1.4.11 

1.5.5 

2.1.1 

2.1.2 

2.1.3 

2.2.1 

2.2.8 

2.3.4 

2.3.6 

2.3.7 

2.3.9 

2.3.10 

2.3.12 

2.3.13 

2.3.14 

2.4.1 

2.4.4 

2.5.2 

2.5.4 

2.5.5 

2.6.1 

2.6.2 

3.2.6 

3.3.2 

3.4.1 

3.4.6 

4.1.2 

4.1.8 

4.2.3 

4.2.4 

4.3.2 

5.1.4 
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Needs  Assessment 

See  Educational  ueeds  assess 

New  York  City 

1.5.5  2.2.8   4.1.6   4.2.2 
4.3.1 

New  York  State 
1.4.9   4.2.10 

Ohio 

4.2.10 

Operations  analysis 

2.3.14  4.2.4   4.3.2   5.1.4 

Oregon 

2.2.3  5.1.1 

Organizations j  theories  of 

1.3.6  1.3.12  1.4.3   3.1.4 

3.4.1  3.4.5   4.1.4   4.1.10 

4.2.2  4.3.1   4.3.4   5.14 

Output  indicators 

1.1.4  1.1.6  1.2.1  1.3.15 
1.3.18  1.4.2  1.4.4  1.4.11 
1.4.12  1.4.13  2.2.7  2.5.3 
2.5.4  3.1.3  3.4.2  3.4.5 
5.15 

Pennsylvania 
1.2.1   1.4.2 

Personal  probability 
1.3.21 


Philadelphia 


1.3.14 
2.4.1 


2.1.1 
2.4.4 


2.3.13  2.3.14 


Policy  analysis 

2.2.9   4.1.12   4.1.13 

Program-Planning  Budgeting  Systems 
1.3.3   1.3.14  1.4.7   1.4.11 


1.4.13 

1.5.1 

1.5.5 

2.1.4 

2.2.1 

2.2.2 

2.2.3 

2.2.4 

2.2.5 

2.2.6 

2.2.7 

2.2.8 

2.2.9 

2.3.13 

2.4.4 

2.4.5 

2.4.7 

2.4.8 

3.2.2 

3.4.1 

4.1.2 

4.1.3 

4.1.11 

4.1.12 

4.1.13 

4.2.1 

4.2.2 

4.2.5 

4.2.7 

4.2.8 

4.2.10 

4.2.11 

4.2.14 

4.3.1 

5.3 

5.6 

5.10 

5.11 

5.12 

5.13 

5.14 

•oject  management 

4.2.3 

Public  opinion  and  the  schools 


1.2.1 

1.3.1 

1.3.2 

1.3.5 

1.3.6 

1.3.16 

1.3.18 

1.4.2 

1.4.9 

1.4.11 

1.4.12 

1.5.2 

1.5.4 

2.1.3 

2.5.2 

3.2.6 

3.4.1 

3.4.6 

4.1.1 

4.1.2 

4.1.  5 

4.1.6 

4.1.7 

4.1.8 

4.1.9 

4.1.10 

4.1.14 

Pupil-event  analysis 
1.1.3 

Q-sort  technique 
1.2.2   1.3.7 
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Regression  analysis 

1.4.4   1.4.11  2.5.3   2.5.4 
5.14 


Relative  utility 
1.3.15 

Resource  allocation 

1.3.19  1.5.2  1.5.5  2.1.2 

2.2.1  2.2.4  2.2.8  2.3.13 

4.1.3  4.2.8  4.2.11  5.9 
5.14 

Scaling  theory 
1.3.20 

School  boards 


Schools,  urban 


1.2.3 
4.1.6 


1.5.2 

4.1.9 


3.4.6 
5.5 


4.1.5 


School  districts 

1.2.3  1.3.3  1.4.13  1.5.2 

1.5.4  2.1.4  2.1.5  2.2.7 
2.3.13  2.3.14  2.4.1  2.4.6 
3.2.2  3.4.6  3.4.7  4.1.5 
4.1.8  4.1.9  4.2.14  5.5 

Schools,  elementary 

1.1.2   1.2.2   1.3.7   2.5.2 

Schools,  rural 
1.1.2   5.1 

Schools,  secondary 

1.1.5  1.3.7 

Schools,  suburban 

1.1.5 


1.3.1 

2.1.2 

2.1.3 

2.3.4 

2.3.5 

2.3.14 

2.4.7 

2.5.2 

3.1.3 

3.2.4 

3.2.6 

3.3.2 

3.4.1 

3.4.7 

4.1.2 

4.1.6 

4.1.8 

4.1.9 

4.2.8 

Similarity  measures 
1.3.13 


Social  change 

1.3.1  1.4.3 
3.2.6  3.3.2 
4.1.13  5.1 


2.3.4  2.3.10 
3.4.2  4.1.10 
5.14 


Statistics,  theory  of 


1.3.21  2.2.4 
2.5.5   2.6.1 

Student  alienation 
1.1.2   1.1.5 

Students,  middle-class 
1.1.5 

Superintendents 

4.1.5 


2.4.3   2.5.2 
5.12    5.14 


Systems  analysis 

1.3.15  1.3.18 

2.3.2  2.3.9 

2.3.14  2.5.1 

3.1.2  3.4.2 

5.9  5.11 


Tennessee 
4.2.10 


2.2.1  2.2.3 

2.3.10  2.3.11 

2.5.3  2.6.2 

4.1.9  4.1.13 
5.14 
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Texas 

1.2.4   5.8 

Transportation 

1.4.10  2.3.1 
5.14 


2.3.10   5.10 


Uncertainty 

1.3.10  2.3.8   2.6.1   5.14 

Universities 

1.3.13  2.2.1   5.9 

Utility 

1.3.1  1.3.4  1.3.9  1.3.10 

1.3.15  1.3.17  1.3.18  1.3.19 

1.4.4  2.3.6  5.7  5.14 
5.16 


Virginia 
1.1.6 

Vocational  education 
2.2.4   2.3.13 

Wisconsin 
4.2.10 

Zero-sum  logic 
1.3.3 
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